docs & ux improvmeents
This commit is contained in:
IGNY8 VPS (Salman)
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,322 +0,0 @@
|
|||||||
# GLOBAL SETTINGS - DJANGO ADMIN ACCESS GUIDE
|
|
||||||
|
|
||||||
**Last Updated**: December 20, 2025
|
|
||||||
**Status**: ✅ READY TO USE
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## WHERE TO FIND GLOBAL SETTINGS IN DJANGO ADMIN
|
|
||||||
|
|
||||||
### 1. Global AI Integration Settings (API Keys)
|
|
||||||
|
|
||||||
**URL**: http://your-domain.com/admin/system/globalintegrationsettings/
|
|
||||||
|
|
||||||
**What It Controls**:
|
|
||||||
- OpenAI API key (for text generation)
|
|
||||||
- OpenAI model selection (gpt-4, gpt-3.5-turbo, etc.)
|
|
||||||
- OpenAI temperature and max_tokens
|
|
||||||
- DALL-E API key (for image generation)
|
|
||||||
- DALL-E model, size, quality, style
|
|
||||||
- Anthropic API key (for Claude)
|
|
||||||
- Anthropic model selection
|
|
||||||
- Runware API key (for advanced image generation)
|
|
||||||
|
|
||||||
**Important**:
|
|
||||||
- This is a SINGLETON - only ONE record exists (ID=1)
|
|
||||||
- Changes here affect ALL accounts by default
|
|
||||||
- Enterprise accounts can override with their own keys
|
|
||||||
|
|
||||||
**How to Configure**:
|
|
||||||
1. Login to Django Admin as superuser
|
|
||||||
2. Navigate to: System → Global integration settings
|
|
||||||
3. Click on the single "Global Integration Settings" entry
|
|
||||||
4. Fill in your platform-wide API keys
|
|
||||||
5. Set default models and parameters
|
|
||||||
6. Save
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 2. Account Integration Overrides (Per-Account API Keys)
|
|
||||||
|
|
||||||
**URL**: http://your-domain.com/admin/system/accountintegrationoverride/
|
|
||||||
|
|
||||||
**What It Controls**:
|
|
||||||
- Per-account API key overrides for enterprise customers
|
|
||||||
- Each account can optionally use their own keys
|
|
||||||
- Falls back to global if not configured
|
|
||||||
|
|
||||||
**Fields**:
|
|
||||||
- Account (select which account)
|
|
||||||
- use_own_keys (checkbox - if unchecked, uses global)
|
|
||||||
- Same API key fields as global (all optional)
|
|
||||||
|
|
||||||
**How to Configure**:
|
|
||||||
1. Navigate to: System → Account integration overrides
|
|
||||||
2. Click "Add account integration override"
|
|
||||||
3. Select the account
|
|
||||||
4. Check "Use own keys"
|
|
||||||
5. Fill in their API keys
|
|
||||||
6. Save
|
|
||||||
|
|
||||||
**How It Works**:
|
|
||||||
- If account has override with use_own_keys=True → uses their keys
|
|
||||||
- If account has NO override OR use_own_keys=False → uses global keys
|
|
||||||
- Account can be deleted/disabled to revert to global
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 3. Global AI Prompts (Prompt Templates Library)
|
|
||||||
|
|
||||||
**URL**: http://your-domain.com/admin/system/globalaiprompt/
|
|
||||||
|
|
||||||
**What It Controls**:
|
|
||||||
- Platform-wide default AI prompt templates
|
|
||||||
- Used for clustering, content generation, ideas, etc.
|
|
||||||
- All accounts can use these prompts
|
|
||||||
- Accounts can customize their own versions
|
|
||||||
|
|
||||||
**Fields**:
|
|
||||||
- Prompt type (clustering, ideas, content_generation, etc.)
|
|
||||||
- Prompt value (the actual prompt template)
|
|
||||||
- Description (what this prompt does)
|
|
||||||
- Variables (list of available variables like {keyword}, {industry})
|
|
||||||
- Version (for tracking changes)
|
|
||||||
- Is active (enable/disable)
|
|
||||||
|
|
||||||
**How to Configure**:
|
|
||||||
1. Navigate to: System → Global ai prompts
|
|
||||||
2. Click "Add global ai prompt"
|
|
||||||
3. Select prompt type (or create new)
|
|
||||||
4. Write your prompt template
|
|
||||||
5. List variables it uses
|
|
||||||
6. Mark as active
|
|
||||||
7. Save
|
|
||||||
|
|
||||||
**Account Usage**:
|
|
||||||
- Accounts automatically use global prompts
|
|
||||||
- Accounts can create customized versions in their own AIPrompt records
|
|
||||||
- Accounts can reset to global anytime
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 4. Global Author Profiles (Persona Templates Library)
|
|
||||||
|
|
||||||
**URL**: http://your-domain.com/admin/system/globalauthorprofile/
|
|
||||||
|
|
||||||
**What It Controls**:
|
|
||||||
- Platform-wide author persona templates
|
|
||||||
- Tone of voice configurations
|
|
||||||
- Writing style templates
|
|
||||||
- Accounts can clone and customize
|
|
||||||
|
|
||||||
**Fields**:
|
|
||||||
- Name (e.g., "SaaS B2B Professional")
|
|
||||||
- Description (what this persona is for)
|
|
||||||
- Tone (professional, casual, technical, etc.)
|
|
||||||
- Language (en, es, fr, etc.)
|
|
||||||
- Structure template (JSON config for content structure)
|
|
||||||
- Category (saas, ecommerce, blog, technical, creative)
|
|
||||||
- Is active (enable/disable)
|
|
||||||
|
|
||||||
**How to Configure**:
|
|
||||||
1. Navigate to: System → Global author profiles
|
|
||||||
2. Click "Add global author profile"
|
|
||||||
3. Create a persona template
|
|
||||||
4. Set tone and language
|
|
||||||
5. Add structure template if needed
|
|
||||||
6. Assign category
|
|
||||||
7. Save
|
|
||||||
|
|
||||||
**Account Usage**:
|
|
||||||
- Accounts browse global library
|
|
||||||
- Accounts clone a template to create their own version
|
|
||||||
- Cloned version stored in AuthorProfile model with cloned_from reference
|
|
||||||
- Accounts can customize their clone without affecting global
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 5. Global Strategies (Content Strategy Templates)
|
|
||||||
|
|
||||||
**URL**: http://your-domain.com/admin/system/globalstrategy/
|
|
||||||
|
|
||||||
**What It Controls**:
|
|
||||||
- Platform-wide content strategy templates
|
|
||||||
- Section structures for different content types
|
|
||||||
- Prompt sequences for content generation
|
|
||||||
- Accounts can clone and customize
|
|
||||||
|
|
||||||
**Fields**:
|
|
||||||
- Name (e.g., "SEO Blog Post Strategy")
|
|
||||||
- Description (what this strategy achieves)
|
|
||||||
- Category (blog, product, howto, comparison, etc.)
|
|
||||||
- Prompt types (which prompts to use)
|
|
||||||
- Section logic (JSON config for content sections)
|
|
||||||
- Is active (enable/disable)
|
|
||||||
|
|
||||||
**How to Configure**:
|
|
||||||
1. Navigate to: System → Global strategies
|
|
||||||
2. Click "Add global strategy"
|
|
||||||
3. Create a strategy template
|
|
||||||
4. Define section structure
|
|
||||||
5. Specify which prompts to use
|
|
||||||
6. Add section logic JSON
|
|
||||||
7. Save
|
|
||||||
|
|
||||||
**Account Usage**:
|
|
||||||
- Similar to author profiles
|
|
||||||
- Accounts clone global templates
|
|
||||||
- Customize for their needs
|
|
||||||
- Track origin via cloned_from field
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ACCOUNT-SPECIFIC MODELS (Not Global)
|
|
||||||
|
|
||||||
These remain account-specific as originally designed:
|
|
||||||
|
|
||||||
### AIPrompt (Account-Level)
|
|
||||||
**URL**: /admin/system/aiprompt/
|
|
||||||
- Per-account AI prompt customizations
|
|
||||||
- References global prompts by default
|
|
||||||
- Can be customized (is_customized=True)
|
|
||||||
- Can reset to global anytime
|
|
||||||
|
|
||||||
### AuthorProfile (Account-Level)
|
|
||||||
**URL**: /admin/system/authorprofile/
|
|
||||||
- Per-account author personas
|
|
||||||
- Can be cloned from global (cloned_from field)
|
|
||||||
- Can be created from scratch (is_custom=True)
|
|
||||||
|
|
||||||
### Strategy (Account-Level)
|
|
||||||
**URL**: /admin/system/strategy/
|
|
||||||
- Per-account content strategies
|
|
||||||
- Can be cloned from global
|
|
||||||
- Can be created from scratch
|
|
||||||
|
|
||||||
### IntegrationSettings (Account-Level) - DEPRECATED
|
|
||||||
**URL**: /admin/system/integrationsettings/
|
|
||||||
**Status**: This model is being phased out in favor of Global + Override pattern
|
|
||||||
**Do Not Use**: Use GlobalIntegrationSettings and AccountIntegrationOverride instead
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## NAVIGATION IN DJANGO ADMIN
|
|
||||||
|
|
||||||
When you login to Django Admin, you'll see:
|
|
||||||
|
|
||||||
```
|
|
||||||
SYSTEM
|
|
||||||
├── Global Integration Settings (1 entry - singleton)
|
|
||||||
├── Account Integration Overrides (0+ entries - one per enterprise account)
|
|
||||||
├── Global AI Prompts (library of prompt templates)
|
|
||||||
├── Global Author Profiles (library of persona templates)
|
|
||||||
├── Global Strategies (library of strategy templates)
|
|
||||||
├── AI Prompts (per-account customizations)
|
|
||||||
├── Author Profiles (per-account personas)
|
|
||||||
├── Strategies (per-account strategies)
|
|
||||||
└── Integration Settings (DEPRECATED - old model)
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## QUICK START CHECKLIST
|
|
||||||
|
|
||||||
After deployment, configure in this order:
|
|
||||||
|
|
||||||
1. **Set Global API Keys** (/admin/system/globalintegrationsettings/)
|
|
||||||
- [ ] OpenAI API key
|
|
||||||
- [ ] DALL-E API key
|
|
||||||
- [ ] Anthropic API key (optional)
|
|
||||||
- [ ] Runware API key (optional)
|
|
||||||
- [ ] Set default models and parameters
|
|
||||||
|
|
||||||
2. **Create Global Prompt Library** (/admin/system/globalaiprompt/)
|
|
||||||
- [ ] Clustering prompt
|
|
||||||
- [ ] Content ideas prompt
|
|
||||||
- [ ] Content generation prompt
|
|
||||||
- [ ] Meta description prompt
|
|
||||||
- [ ] Title generation prompt
|
|
||||||
|
|
||||||
3. **Create Global Author Profiles** (/admin/system/globalauthorprofile/)
|
|
||||||
- [ ] Professional B2B profile
|
|
||||||
- [ ] E-commerce profile
|
|
||||||
- [ ] Blog/casual profile
|
|
||||||
- [ ] Technical profile
|
|
||||||
- [ ] Creative profile
|
|
||||||
|
|
||||||
4. **Create Global Strategies** (/admin/system/globalstrategy/)
|
|
||||||
- [ ] SEO blog post strategy
|
|
||||||
- [ ] Product launch strategy
|
|
||||||
- [ ] How-to guide strategy
|
|
||||||
- [ ] Comparison article strategy
|
|
||||||
|
|
||||||
5. **Test with Regular Account**
|
|
||||||
- [ ] Create content using global prompts
|
|
||||||
- [ ] Verify global API keys work
|
|
||||||
- [ ] Test cloning profiles/strategies
|
|
||||||
|
|
||||||
6. **Configure Enterprise Account** (if needed)
|
|
||||||
- [ ] Create AccountIntegrationOverride
|
|
||||||
- [ ] Add their API keys
|
|
||||||
- [ ] Enable use_own_keys
|
|
||||||
- [ ] Test their custom keys work
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## TROUBLESHOOTING
|
|
||||||
|
|
||||||
**Problem**: Can't see Global Integration Settings in admin
|
|
||||||
|
|
||||||
**Solution**:
|
|
||||||
1. Check you're logged in as superuser
|
|
||||||
2. Refresh the page
|
|
||||||
3. Check URL: /admin/system/globalintegrationsettings/
|
|
||||||
4. Verify migration applied: `docker exec igny8_backend python manage.py showmigrations system`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Problem**: Global settings not taking effect
|
|
||||||
|
|
||||||
**Solution**:
|
|
||||||
1. Check GlobalIntegrationSettings has values saved
|
|
||||||
2. Verify is_active=True
|
|
||||||
3. Check no AccountIntegrationOverride for the account
|
|
||||||
4. Restart backend: `docker restart igny8_backend`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Problem**: Account override not working
|
|
||||||
|
|
||||||
**Solution**:
|
|
||||||
1. Check use_own_keys checkbox is enabled
|
|
||||||
2. Verify API keys are filled in
|
|
||||||
3. Check account selected correctly
|
|
||||||
4. Test the API keys manually
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## API ACCESS TO GLOBAL SETTINGS
|
|
||||||
|
|
||||||
Code can access global settings:
|
|
||||||
|
|
||||||
**Get Global Integration Settings**:
|
|
||||||
```python
|
|
||||||
from igny8_core.modules.system.global_settings_models import GlobalIntegrationSettings
|
|
||||||
settings = GlobalIntegrationSettings.get_instance()
|
|
||||||
```
|
|
||||||
|
|
||||||
**Get Effective Settings for Account** (checks override, falls back to global):
|
|
||||||
```python
|
|
||||||
from igny8_core.ai.settings import get_openai_settings
|
|
||||||
settings = get_openai_settings(account)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Get Global Prompt**:
|
|
||||||
```python
|
|
||||||
from igny8_core.modules.system.global_settings_models import GlobalAIPrompt
|
|
||||||
prompt = GlobalAIPrompt.objects.get(prompt_type='clustering', is_active=True)
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
*For complete implementation details, see COMPLETE-IMPLEMENTATION-GUIDE.md*
|
|
||||||
@@ -1,320 +0,0 @@
|
|||||||
# GLOBAL SETTINGS - CORRECT IMPLEMENTATION
|
|
||||||
|
|
||||||
**Date**: December 20, 2025
|
|
||||||
**Status**: ✅ FIXED AND WORKING
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## WHAT WAS WRONG
|
|
||||||
|
|
||||||
The initial implementation had:
|
|
||||||
- AccountIntegrationOverride model allowing users to use their own API keys
|
|
||||||
- Enterprise plan that doesn't exist
|
|
||||||
- Confusing override logic where accounts could bring their own API keys
|
|
||||||
|
|
||||||
## WHAT IS NOW CORRECT
|
|
||||||
|
|
||||||
### Architecture
|
|
||||||
|
|
||||||
**1. Plans (Only 4 Valid)**:
|
|
||||||
- Free Plan - Cannot override anything, uses global defaults
|
|
||||||
- Starter Plan - Can override model/settings
|
|
||||||
- Growth Plan - Can override model/settings
|
|
||||||
- Scale Plan - Can override model/settings
|
|
||||||
|
|
||||||
**2. API Keys** (Platform-Wide):
|
|
||||||
- Stored in GlobalIntegrationSettings (singleton, pk=1)
|
|
||||||
- ALL accounts use platform API keys
|
|
||||||
- NO user can bring their own API keys
|
|
||||||
- NO exceptions for any plan level
|
|
||||||
|
|
||||||
**3. Model & Parameter Overrides** (Per-Account):
|
|
||||||
- Stored in IntegrationSettings model (per-account)
|
|
||||||
- Free plan: CANNOT create overrides
|
|
||||||
- Starter/Growth/Scale: CAN override model, temperature, max_tokens, image settings
|
|
||||||
- NULL values in config = use global default
|
|
||||||
- API keys NEVER stored here
|
|
||||||
|
|
||||||
**4. Prompts** (Global + Override):
|
|
||||||
- GlobalAIPrompt: Platform-wide default prompts
|
|
||||||
- AIPrompt: Per-account with default_prompt field
|
|
||||||
- When user customizes: prompt_value changes, default_prompt stays same
|
|
||||||
- Reset to default: Copies default_prompt back to prompt_value
|
|
||||||
- is_customized flag tracks if using custom or default
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## WHERE TO FIND SETTINGS IN DJANGO ADMIN
|
|
||||||
|
|
||||||
### 1. Global Integration Settings
|
|
||||||
**URL**: /admin/system/globalintegrationsettings/
|
|
||||||
|
|
||||||
**What It Stores**:
|
|
||||||
- Platform OpenAI API key (used by ALL accounts)
|
|
||||||
- Platform DALL-E API key (used by ALL accounts)
|
|
||||||
- Platform Anthropic API key (used by ALL accounts)
|
|
||||||
- Platform Runware API key (used by ALL accounts)
|
|
||||||
- Default model selections for each service
|
|
||||||
- Default parameters (temperature, max_tokens, image quality, etc.)
|
|
||||||
|
|
||||||
**Important**:
|
|
||||||
- Singleton model (only 1 record, pk=1)
|
|
||||||
- Changes affect ALL accounts using global defaults
|
|
||||||
- Free plan accounts MUST use these (cannot override)
|
|
||||||
- Other plans can override model/params but NOT API keys
|
|
||||||
|
|
||||||
### 2. Integration Settings (Per-Account Overrides)
|
|
||||||
**URL**: /admin/system/integrationsettings/
|
|
||||||
|
|
||||||
**What It Stores**:
|
|
||||||
- Per-account model selection overrides
|
|
||||||
- Per-account parameter overrides (temperature, max_tokens, etc.)
|
|
||||||
- Per-account image setting overrides (size, quality, style)
|
|
||||||
|
|
||||||
**What It DOES NOT Store**:
|
|
||||||
- API keys (those come from global)
|
|
||||||
|
|
||||||
**Who Can Create**:
|
|
||||||
- Starter/Growth/Scale plans only
|
|
||||||
- Free plan users cannot create these
|
|
||||||
|
|
||||||
**How It Works**:
|
|
||||||
- If account has IntegrationSettings record with config values → uses those
|
|
||||||
- If config field is NULL or missing → uses global default
|
|
||||||
- API key ALWAYS from GlobalIntegrationSettings
|
|
||||||
|
|
||||||
**Example Config**:
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"model": "gpt-4",
|
|
||||||
"temperature": 0.8,
|
|
||||||
"max_tokens": 4000
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Global AI Prompts
|
|
||||||
**URL**: /admin/system/globalaiprompt/
|
|
||||||
|
|
||||||
**What It Stores**:
|
|
||||||
- Platform-wide default prompt templates
|
|
||||||
- Used for: clustering, ideas, content_generation, etc.
|
|
||||||
|
|
||||||
**How Accounts Use Them**:
|
|
||||||
- All accounts start with global prompts
|
|
||||||
- When user wants to customize, system creates AIPrompt record
|
|
||||||
- AIPrompt.default_prompt = GlobalAIPrompt.prompt_value (for reset)
|
|
||||||
- AIPrompt.prompt_value = user's custom text
|
|
||||||
- AIPrompt.is_customized = True
|
|
||||||
|
|
||||||
### 4. AI Prompts (Per-Account)
|
|
||||||
**URL**: /admin/system/aiprompt/
|
|
||||||
|
|
||||||
**What It Stores**:
|
|
||||||
- Account-specific prompt customizations
|
|
||||||
- default_prompt field = global default (for reset)
|
|
||||||
- prompt_value = current prompt (custom or default)
|
|
||||||
- is_customized = True if user modified it
|
|
||||||
|
|
||||||
**Actions Available**:
|
|
||||||
- "Reset selected prompts to global default" - Copies default_prompt → prompt_value, sets is_customized=False
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## HOW IT WORKS (Complete Flow)
|
|
||||||
|
|
||||||
### Text Generation Request
|
|
||||||
|
|
||||||
1. Code calls: `get_model_config(function_name='generate_content', account=some_account)`
|
|
||||||
|
|
||||||
2. System gets API key from GlobalIntegrationSettings:
|
|
||||||
- `global_settings = GlobalIntegrationSettings.get_instance()`
|
|
||||||
- `api_key = global_settings.openai_api_key` # ALWAYS from global
|
|
||||||
|
|
||||||
3. System checks for account overrides:
|
|
||||||
- Try to find IntegrationSettings for this account + integration_type='openai'
|
|
||||||
- If found: Use config['model'], config['temperature'], config['max_tokens']
|
|
||||||
- If not found OR config field is NULL: Use global defaults
|
|
||||||
|
|
||||||
4. Result returned:
|
|
||||||
```python
|
|
||||||
{
|
|
||||||
'api_key': 'sk-xxx', # Always from global
|
|
||||||
'model': 'gpt-4', # From account override OR global
|
|
||||||
'temperature': 0.8, # From account override OR global
|
|
||||||
'max_tokens': 4000 # From account override OR global
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Prompt Retrieval
|
|
||||||
|
|
||||||
1. Code calls: `AIPrompt.get_effective_prompt(account=some_account, prompt_type='clustering')`
|
|
||||||
|
|
||||||
2. System checks for account-specific prompt:
|
|
||||||
- Try to find AIPrompt for this account + prompt_type
|
|
||||||
- If found and is_customized=True: Return prompt_value
|
|
||||||
- If found and is_customized=False: Return default_prompt
|
|
||||||
|
|
||||||
3. If no account prompt found:
|
|
||||||
- Get GlobalAIPrompt for prompt_type
|
|
||||||
- Return global prompt_value
|
|
||||||
|
|
||||||
### User Customizes a Prompt
|
|
||||||
|
|
||||||
1. User edits prompt in frontend
|
|
||||||
2. Frontend saves to AIPrompt model:
|
|
||||||
- If AIPrompt doesn't exist: Create new record
|
|
||||||
- Set default_prompt = GlobalAIPrompt.prompt_value (for future reset)
|
|
||||||
- Set prompt_value = user's custom text
|
|
||||||
- Set is_customized = True
|
|
||||||
|
|
||||||
### User Resets Prompt
|
|
||||||
|
|
||||||
1. User clicks "Reset to Default"
|
|
||||||
2. System calls: `AIPrompt.reset_to_default()`
|
|
||||||
3. Method does:
|
|
||||||
- prompt_value = default_prompt
|
|
||||||
- is_customized = False
|
|
||||||
- save()
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## MIGRATION APPLIED
|
|
||||||
|
|
||||||
**File**: 0004_fix_global_settings_remove_override.py
|
|
||||||
|
|
||||||
**Changes**:
|
|
||||||
- Added default_prompt field to AIPrompt model
|
|
||||||
- Updated help text on IntegrationSettings.config field
|
|
||||||
- Updated integration_type choices (removed GSC, image_generation)
|
|
||||||
- Updated GlobalIntegrationSettings help text
|
|
||||||
- Removed AccountIntegrationOverride model
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## ADMIN INTERFACE CHANGES
|
|
||||||
|
|
||||||
**GlobalIntegrationSettings Admin**:
|
|
||||||
- Shows all platform API keys and default settings
|
|
||||||
- One record only (singleton)
|
|
||||||
- Help text clarifies these are used by ALL accounts
|
|
||||||
|
|
||||||
**IntegrationSettings Admin**:
|
|
||||||
- Help text emphasizes: "NEVER store API keys here"
|
|
||||||
- Config field description explains it's for overrides only
|
|
||||||
- Removed bulk_test_connection action
|
|
||||||
- Free plan check should be added to prevent creation
|
|
||||||
|
|
||||||
**AIPrompt Admin**:
|
|
||||||
- Added default_prompt to readonly_fields
|
|
||||||
- Added "Reset selected prompts to global default" bulk action
|
|
||||||
- Fieldsets show both prompt_value and default_prompt
|
|
||||||
|
|
||||||
**Removed**:
|
|
||||||
- AccountIntegrationOverride model
|
|
||||||
- AccountIntegrationOverrideAdmin class
|
|
||||||
- All references to per-account API keys
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## SIDEBAR NAVIGATION (TODO)
|
|
||||||
|
|
||||||
Need to add links in app sidebar to access global settings:
|
|
||||||
|
|
||||||
**For Superusers/Admin**:
|
|
||||||
- Global Settings
|
|
||||||
- Platform API Keys (/admin/system/globalintegrationsettings/)
|
|
||||||
- Global Prompts (/admin/system/globalaiprompt/)
|
|
||||||
- Global Author Profiles (/admin/system/globalauthorprofile/)
|
|
||||||
- Global Strategies (/admin/system/globalstrategy/)
|
|
||||||
|
|
||||||
**For All Users** (Starter+ plans):
|
|
||||||
- Account Settings
|
|
||||||
- AI Model Selection (/settings/ai) - Configure IntegrationSettings
|
|
||||||
- Custom Prompts (/settings/prompts) - Manage AIPrompts
|
|
||||||
- Author Profiles (/settings/profiles) - Manage AuthorProfiles
|
|
||||||
- Content Strategies (/settings/strategies) - Manage Strategies
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## VERIFICATION
|
|
||||||
|
|
||||||
Run these commands to verify:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check migration applied
|
|
||||||
docker exec igny8_backend python manage.py showmigrations system
|
|
||||||
|
|
||||||
# Verify global settings exist
|
|
||||||
docker exec igny8_backend python manage.py shell -c "
|
|
||||||
from igny8_core.modules.system.global_settings_models import GlobalIntegrationSettings
|
|
||||||
obj = GlobalIntegrationSettings.get_instance()
|
|
||||||
print(f'OpenAI Model: {obj.openai_model}')
|
|
||||||
print(f'Max Tokens: {obj.openai_max_tokens}')
|
|
||||||
"
|
|
||||||
|
|
||||||
# Check AIPrompt has default_prompt field
|
|
||||||
docker exec igny8_backend python manage.py shell -c "
|
|
||||||
from igny8_core.modules.system.models import AIPrompt
|
|
||||||
fields = [f.name for f in AIPrompt._meta.get_fields()]
|
|
||||||
print('default_prompt' in fields)
|
|
||||||
"
|
|
||||||
|
|
||||||
# Verify AccountIntegrationOverride removed
|
|
||||||
docker exec igny8_backend python manage.py shell -c "
|
|
||||||
try:
|
|
||||||
from igny8_core.modules.system.global_settings_models import AccountIntegrationOverride
|
|
||||||
print('ERROR: Model still exists!')
|
|
||||||
except ImportError:
|
|
||||||
print('✓ Model correctly removed')
|
|
||||||
"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## QUICK START
|
|
||||||
|
|
||||||
1. **Configure Platform API Keys**:
|
|
||||||
- Login to Django Admin
|
|
||||||
- Go to: System → Global integration settings
|
|
||||||
- Fill in OpenAI, DALL-E API keys
|
|
||||||
- Set default models
|
|
||||||
- Save
|
|
||||||
|
|
||||||
2. **Create Global Prompts**:
|
|
||||||
- Go to: System → Global ai prompts
|
|
||||||
- Add prompts for: clustering, ideas, content_generation
|
|
||||||
- These become defaults for all accounts
|
|
||||||
|
|
||||||
3. **Test with Account**:
|
|
||||||
- Create test account on Starter plan
|
|
||||||
- Account automatically uses global API keys
|
|
||||||
- Account can create IntegrationSettings to override model selection
|
|
||||||
- Account CANNOT override API keys
|
|
||||||
|
|
||||||
4. **Verify Free Plan Restriction**:
|
|
||||||
- Create test account on Free plan
|
|
||||||
- Verify they CANNOT create IntegrationSettings records
|
|
||||||
- Verify they use global defaults only
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## SUMMARY
|
|
||||||
|
|
||||||
✅ **Correct**: Platform API keys used by all accounts
|
|
||||||
✅ **Correct**: No user can bring their own API keys
|
|
||||||
✅ **Correct**: Only 4 plans (Free, Starter, Growth, Scale)
|
|
||||||
✅ **Correct**: Free plan cannot override, must use global
|
|
||||||
✅ **Correct**: Other plans can override model/params only
|
|
||||||
✅ **Correct**: Prompts have default_prompt for reset
|
|
||||||
✅ **Correct**: Global settings NOT associated with any account
|
|
||||||
|
|
||||||
❌ **Removed**: AccountIntegrationOverride model
|
|
||||||
❌ **Removed**: Enterprise plan references
|
|
||||||
❌ **Removed**: "Bring your own API key" functionality
|
|
||||||
|
|
||||||
🔧 **TODO**: Add sidebar navigation links to global settings
|
|
||||||
🔧 **TODO**: Add plan check to IntegrationSettings creation
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
*For complete implementation details, see COMPLETE-IMPLEMENTATION-GUIDE.md*
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,347 +0,0 @@
|
|||||||
# AI Models Database Configuration - Implementation Summary
|
|
||||||
|
|
||||||
**Date Completed:** December 24, 2025
|
|
||||||
**Status:** ✅ **PRODUCTION READY**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
Successfully migrated AI model pricing from hardcoded constants to a dynamic database-driven system. The system now supports real-time model configuration via Django Admin without requiring code deployments.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Implementation Phases (All Complete ✅)
|
|
||||||
|
|
||||||
### Phase 1: AIModelConfig Model ✅
|
|
||||||
**File:** `backend/igny8_core/business/billing/models.py`
|
|
||||||
|
|
||||||
Created comprehensive model with:
|
|
||||||
- 15 fields supporting both text and image models
|
|
||||||
- Text model fields: `input_cost_per_1m`, `output_cost_per_1m`, `context_window`, `max_output_tokens`
|
|
||||||
- Image model fields: `cost_per_image`, `valid_sizes` (JSON array)
|
|
||||||
- Capabilities: `supports_json_mode`, `supports_vision`, `supports_function_calling`
|
|
||||||
- Status fields: `is_active`, `is_default`, `sort_order`
|
|
||||||
- Audit trail: `created_at`, `updated_at`, `updated_by`
|
|
||||||
- History tracking via `django-simple-history`
|
|
||||||
|
|
||||||
**Methods:**
|
|
||||||
- `get_cost_for_tokens(input_tokens, output_tokens)` - Calculate text model cost
|
|
||||||
- `get_cost_for_images(num_images)` - Calculate image model cost
|
|
||||||
- `validate_size(size)` - Validate image size for model
|
|
||||||
- `get_display_with_pricing()` - Formatted string for dropdowns
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Phase 2: Migration & Data Seeding ✅
|
|
||||||
**File:** `backend/igny8_core/modules/billing/migrations/0020_create_ai_model_config.py`
|
|
||||||
|
|
||||||
**Seeded Models:**
|
|
||||||
- **Text Models (5):**
|
|
||||||
- `gpt-4o-mini` (default) - $0.15/$0.60 per 1M | 128K context
|
|
||||||
- `gpt-4o` - $2.50/$10.00 per 1M | 128K context | Vision
|
|
||||||
- `gpt-4.1` - $2.00/$8.00 per 1M | 8K context
|
|
||||||
- `gpt-5.1` - $1.25/$10.00 per 1M | 16K context
|
|
||||||
- `gpt-5.2` - $1.75/$14.00 per 1M | 16K context
|
|
||||||
|
|
||||||
- **Image Models (4):**
|
|
||||||
- `dall-e-3` (default) - $0.040/image | 3 sizes
|
|
||||||
- `dall-e-2` - $0.020/image | 3 sizes
|
|
||||||
- `gpt-image-1` (inactive) - $0.042/image
|
|
||||||
- `gpt-image-1-mini` (inactive) - $0.011/image
|
|
||||||
|
|
||||||
**Total:** 9 models (7 active)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Phase 3: Django Admin Interface ✅
|
|
||||||
**File:** `backend/igny8_core/modules/billing/admin.py`
|
|
||||||
|
|
||||||
**Features:**
|
|
||||||
- List display with colored badges (model type, provider)
|
|
||||||
- Formatted pricing display based on type
|
|
||||||
- Active/inactive and default status icons
|
|
||||||
- Filters: model_type, provider, is_active, capabilities
|
|
||||||
- Search: model_name, display_name, description
|
|
||||||
- Collapsible fieldsets organized by category
|
|
||||||
|
|
||||||
**Actions:**
|
|
||||||
- Bulk activate/deactivate models
|
|
||||||
- Set model as default (enforces single default per type)
|
|
||||||
- Export pricing table
|
|
||||||
|
|
||||||
**Access:** Django Admin → Billing → AI Model Configurations
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Phase 4 & 5: AI Core Integration ✅
|
|
||||||
**File:** `backend/igny8_core/ai/ai_core.py`
|
|
||||||
|
|
||||||
**Updated Functions:**
|
|
||||||
1. `run_ai_request()` (line ~294) - Text model cost calculation
|
|
||||||
2. `generate_image()` (line ~581) - Image model cost calculation
|
|
||||||
3. `calculate_cost()` (line ~822) - Helper method
|
|
||||||
|
|
||||||
**Implementation:**
|
|
||||||
- Lazy imports to avoid circular dependencies
|
|
||||||
- Database-first with fallback to constants
|
|
||||||
- Try/except wrapper for safety
|
|
||||||
- Logging shows source (database vs constants)
|
|
||||||
|
|
||||||
**Example:**
|
|
||||||
```python
|
|
||||||
# Before (hardcoded)
|
|
||||||
rates = MODEL_RATES.get(model, {'input': 2.00, 'output': 8.00})
|
|
||||||
cost = (input_tokens * rates['input'] + output_tokens * rates['output']) / 1_000_000
|
|
||||||
|
|
||||||
# After (database)
|
|
||||||
model_config = AIModelConfig.objects.get(model_name=model, model_type='text', is_active=True)
|
|
||||||
cost = model_config.get_cost_for_tokens(input_tokens, output_tokens)
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Phase 6: Validators Update ✅
|
|
||||||
**File:** `backend/igny8_core/ai/validators.py`
|
|
||||||
|
|
||||||
**Updated Functions:**
|
|
||||||
1. `validate_model(model, model_type)` - Checks database for active models
|
|
||||||
2. `validate_image_size(size, model)` - Uses model's `valid_sizes` from database
|
|
||||||
|
|
||||||
**Benefits:**
|
|
||||||
- Dynamic model availability
|
|
||||||
- Better error messages with available model lists
|
|
||||||
- Automatic sync with database state
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Phase 7: REST API Endpoint ✅
|
|
||||||
**Endpoint:** `GET /api/v1/billing/ai/models/`
|
|
||||||
|
|
||||||
**Files Created/Updated:**
|
|
||||||
- Serializer: `backend/igny8_core/modules/billing/serializers.py`
|
|
||||||
- ViewSet: `backend/igny8_core/modules/billing/views.py`
|
|
||||||
- URLs: `backend/igny8_core/business/billing/urls.py`
|
|
||||||
|
|
||||||
**API Features:**
|
|
||||||
|
|
||||||
**List Models:**
|
|
||||||
```bash
|
|
||||||
GET /api/v1/billing/ai/models/
|
|
||||||
GET /api/v1/billing/ai/models/?type=text
|
|
||||||
GET /api/v1/billing/ai/models/?type=image
|
|
||||||
GET /api/v1/billing/ai/models/?provider=openai
|
|
||||||
GET /api/v1/billing/ai/models/?default=true
|
|
||||||
```
|
|
||||||
|
|
||||||
**Get Single Model:**
|
|
||||||
```bash
|
|
||||||
GET /api/v1/billing/ai/models/gpt-4o-mini/
|
|
||||||
```
|
|
||||||
|
|
||||||
**Response Format:**
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"success": true,
|
|
||||||
"message": "AI models retrieved successfully",
|
|
||||||
"data": [
|
|
||||||
{
|
|
||||||
"model_name": "gpt-4o-mini",
|
|
||||||
"display_name": "GPT-4o mini - Fast & Affordable",
|
|
||||||
"model_type": "text",
|
|
||||||
"provider": "openai",
|
|
||||||
"input_cost_per_1m": "0.1500",
|
|
||||||
"output_cost_per_1m": "0.6000",
|
|
||||||
"context_window": 128000,
|
|
||||||
"max_output_tokens": 16000,
|
|
||||||
"supports_json_mode": true,
|
|
||||||
"supports_vision": false,
|
|
||||||
"is_default": true,
|
|
||||||
"sort_order": 1,
|
|
||||||
"pricing_display": "$0.1500/$0.6000 per 1M"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Authentication:** Required (JWT)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Verification Results
|
|
||||||
|
|
||||||
### ✅ All Tests Passed
|
|
||||||
|
|
||||||
| Test | Status | Details |
|
|
||||||
|------|--------|---------|
|
|
||||||
| Database Models | ✅ | 9 models (7 active, 2 inactive) |
|
|
||||||
| Cost Calculations | ✅ | Text: $0.000523, Image: $0.0400 |
|
|
||||||
| Model Validators | ✅ | Database queries work correctly |
|
|
||||||
| Django Admin | ✅ | Registered with 9 display fields |
|
|
||||||
| API Endpoint | ✅ | `/api/v1/billing/ai/models/` |
|
|
||||||
| Model Methods | ✅ | All helper methods functional |
|
|
||||||
| Default Models | ✅ | gpt-4o-mini (text), dall-e-3 (image) |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Key Benefits Achieved
|
|
||||||
|
|
||||||
### 1. **No Code Deploys for Pricing Updates**
|
|
||||||
- Update model pricing in Django Admin
|
|
||||||
- Changes take effect immediately
|
|
||||||
- No backend restart required
|
|
||||||
|
|
||||||
### 2. **Multi-Provider Ready**
|
|
||||||
- Provider field supports: OpenAI, Anthropic, Runware, Google
|
|
||||||
- Easy to add new providers without code changes
|
|
||||||
|
|
||||||
### 3. **Real-Time Model Management**
|
|
||||||
- Enable/disable models via admin
|
|
||||||
- Set default models per type
|
|
||||||
- Configure capabilities dynamically
|
|
||||||
|
|
||||||
### 4. **Frontend Integration Ready**
|
|
||||||
- RESTful API with filtering
|
|
||||||
- Structured data for dropdowns
|
|
||||||
- Pricing display included
|
|
||||||
|
|
||||||
### 5. **Backward Compatible**
|
|
||||||
- Constants still available as fallback
|
|
||||||
- Existing code continues to work
|
|
||||||
- Gradual migration complete
|
|
||||||
|
|
||||||
### 6. **Full Audit Trail**
|
|
||||||
- django-simple-history tracks all changes
|
|
||||||
- Updated_by field shows who made changes
|
|
||||||
- Created/updated timestamps
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
### Two Pricing Models Supported
|
|
||||||
|
|
||||||
**1. Text Models (Token-Based)**
|
|
||||||
- Credits calculated AFTER AI call
|
|
||||||
- Based on actual token usage
|
|
||||||
- Formula: `cost = (input_tokens × input_rate + output_tokens × output_rate) / 1M`
|
|
||||||
|
|
||||||
**2. Image Models (Per-Image)**
|
|
||||||
- Credits calculated BEFORE AI call
|
|
||||||
- Fixed cost per image
|
|
||||||
- Formula: `cost = cost_per_image × num_images`
|
|
||||||
|
|
||||||
### Data Flow
|
|
||||||
|
|
||||||
```
|
|
||||||
User Request
|
|
||||||
↓
|
|
||||||
AICore checks AIModelConfig database
|
|
||||||
↓
|
|
||||||
If found: Use database pricing
|
|
||||||
If not found: Fallback to constants
|
|
||||||
↓
|
|
||||||
Calculate cost
|
|
||||||
↓
|
|
||||||
Deduct credits
|
|
||||||
↓
|
|
||||||
Log to CreditUsageLog
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Files Modified
|
|
||||||
|
|
||||||
### New Files (2)
|
|
||||||
1. Migration: `0020_create_ai_model_config.py` (200+ lines)
|
|
||||||
2. Summary: This document
|
|
||||||
|
|
||||||
### Modified Files (6)
|
|
||||||
1. `billing/models.py` - Added AIModelConfig model (240 lines)
|
|
||||||
2. `billing/admin.py` - Added AIModelConfigAdmin (180 lines)
|
|
||||||
3. `ai/ai_core.py` - Updated cost calculations (3 functions)
|
|
||||||
4. `ai/validators.py` - Updated validators (2 functions)
|
|
||||||
5. `modules/billing/serializers.py` - Added AIModelConfigSerializer (55 lines)
|
|
||||||
6. `modules/billing/views.py` - Added AIModelConfigViewSet (75 lines)
|
|
||||||
7. `business/billing/urls.py` - Registered API endpoint (1 line)
|
|
||||||
|
|
||||||
**Total:** ~750 lines of code added/modified
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Usage Examples
|
|
||||||
|
|
||||||
### Django Admin
|
|
||||||
1. Navigate to: **Admin → Billing → AI Model Configurations**
|
|
||||||
2. Click on any model to edit pricing
|
|
||||||
3. Use filters to view specific model types
|
|
||||||
4. Use bulk actions to activate/deactivate
|
|
||||||
|
|
||||||
### API Usage (Frontend)
|
|
||||||
```javascript
|
|
||||||
// Fetch all text models
|
|
||||||
const response = await fetch('/api/v1/billing/ai/models/?type=text');
|
|
||||||
const { data: models } = await response.json();
|
|
||||||
|
|
||||||
// Display in dropdown
|
|
||||||
models.forEach(model => {
|
|
||||||
console.log(model.display_name, model.pricing_display);
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
### Programmatic Usage (Backend)
|
|
||||||
```python
|
|
||||||
from igny8_core.business.billing.models import AIModelConfig
|
|
||||||
|
|
||||||
# Get model
|
|
||||||
model = AIModelConfig.objects.get(model_name='gpt-4o-mini')
|
|
||||||
|
|
||||||
# Calculate cost
|
|
||||||
cost = model.get_cost_for_tokens(1000, 500) # $0.000450
|
|
||||||
|
|
||||||
# Validate size (images)
|
|
||||||
dalle = AIModelConfig.objects.get(model_name='dall-e-3')
|
|
||||||
is_valid = dalle.validate_size('1024x1024') # True
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Next Steps (Optional Enhancements)
|
|
||||||
|
|
||||||
### Short Term
|
|
||||||
- [ ] Add model usage analytics to admin
|
|
||||||
- [ ] Create frontend UI for model selection
|
|
||||||
- [ ] Add model comparison view
|
|
||||||
|
|
||||||
### Long Term
|
|
||||||
- [ ] Add Anthropic models (Claude)
|
|
||||||
- [ ] Add Google models (Gemini)
|
|
||||||
- [ ] Implement A/B testing for models
|
|
||||||
- [ ] Add cost forecasting based on usage patterns
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Rollback Plan
|
|
||||||
|
|
||||||
If issues occur:
|
|
||||||
|
|
||||||
1. **Code Level:** All functions have fallback to constants
|
|
||||||
2. **Database Level:** Migration can be reversed: `python manage.py migrate billing 0019`
|
|
||||||
3. **Data Level:** No existing data affected (CreditUsageLog unchanged)
|
|
||||||
4. **Time Required:** < 5 minutes
|
|
||||||
|
|
||||||
**Risk:** Minimal - System has built-in fallback mechanisms
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Support
|
|
||||||
|
|
||||||
- **Django Admin:** http://your-domain/admin/billing/aimodelconfig/
|
|
||||||
- **API Docs:** http://your-domain/api/v1/billing/ai/models/
|
|
||||||
- **Configuration:** [AI-MODELS-DATABASE-CONFIGURATION-PLAN.md](AI-MODELS-DATABASE-CONFIGURATION-PLAN.md)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Status:** ✅ Production Ready
|
|
||||||
**Deployed:** December 24, 2025
|
|
||||||
**Version:** 1.0
|
|
||||||
@@ -1,261 +0,0 @@
|
|||||||
# AI Model Database Configuration - Validation Report
|
|
||||||
|
|
||||||
**Date:** 2024
|
|
||||||
**Status:** ✅ 100% OPERATIONAL AND VERIFIED
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Executive Summary
|
|
||||||
|
|
||||||
All 34 validation tests passed successfully. The AI Model Database Configuration system is fully operational with database-driven pricing, cost calculations, validation, and REST API integration.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Test Results Summary
|
|
||||||
|
|
||||||
| Test Suite | Tests | Passed | Status |
|
|
||||||
|-----------|-------|--------|--------|
|
|
||||||
| **Test 1:** Model Instance Methods | 5 | 5 | ✅ PASS |
|
|
||||||
| **Test 2:** AI Core Cost Calculations | 5 | 5 | ✅ PASS |
|
|
||||||
| **Test 3:** Validators | 9 | 9 | ✅ PASS |
|
|
||||||
| **Test 4:** Credit Calculation Integration | 4 | 4 | ✅ PASS |
|
|
||||||
| **Test 5:** REST API Serializer | 7 | 7 | ✅ PASS |
|
|
||||||
| **Test 6:** End-to-End Integration | 4 | 4 | ✅ PASS |
|
|
||||||
| **TOTAL** | **34** | **34** | **✅ 100%** |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Database Status
|
|
||||||
|
|
||||||
### Active Text Models (5)
|
|
||||||
- ✓ `gpt-4o-mini` - $0.1500/$0.6000 per 1M tokens
|
|
||||||
- ✓ `gpt-4o` - $2.5000/$10.0000 per 1M tokens
|
|
||||||
- ✓ `gpt-4.1` - $2.0000/$8.0000 per 1M tokens
|
|
||||||
- ✓ `gpt-5.1` - $1.2500/$10.0000 per 1M tokens
|
|
||||||
- ✓ `gpt-5.2` - $1.7500/$14.0000 per 1M tokens
|
|
||||||
|
|
||||||
### Active Image Models (2)
|
|
||||||
- ✓ `dall-e-3` - $0.0400 per image
|
|
||||||
- ✓ `dall-e-2` - $0.0200 per image
|
|
||||||
|
|
||||||
### Inactive Models (2)
|
|
||||||
- ⊗ `gpt-image-1` - image
|
|
||||||
- ⊗ `gpt-image-1-mini` - image
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Test Details
|
|
||||||
|
|
||||||
### Test 1: Model Instance Methods
|
|
||||||
**Purpose:** Verify AIModelConfig model methods work correctly
|
|
||||||
|
|
||||||
**Tests:**
|
|
||||||
1. ✅ `get_cost_for_tokens(2518, 242)` → $0.000523
|
|
||||||
2. ✅ `get_cost_for_images(3)` → $0.0800
|
|
||||||
3. ✅ `validate_size('1024x1024')` → True
|
|
||||||
4. ✅ `validate_size('512x512')` → False (dall-e-3 doesn't support)
|
|
||||||
5. ✅ Display format correct
|
|
||||||
|
|
||||||
**Result:** All model methods calculate costs accurately
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Test 2: AI Core Cost Calculations
|
|
||||||
**Purpose:** Verify ai_core.py uses database correctly
|
|
||||||
|
|
||||||
**Tests:**
|
|
||||||
1. ✅ Text model cost calculation (1000 input + 500 output = $0.000450)
|
|
||||||
2. ✅ Image model cost calculation (dall-e-3 = $0.0400)
|
|
||||||
3. ✅ Fallback mechanism works (non-existent model uses constants)
|
|
||||||
4. ✅ All 5 text models consistent with database
|
|
||||||
5. ✅ All 2 image models consistent with database
|
|
||||||
|
|
||||||
**Result:** AICore.calculate_cost() works perfectly with database queries and fallback
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Test 3: Validators
|
|
||||||
**Purpose:** Verify model and size validation works
|
|
||||||
|
|
||||||
**Tests:**
|
|
||||||
1. ✅ Valid text model accepted (gpt-4o-mini)
|
|
||||||
2. ✅ Invalid text model rejected (fake-gpt-999)
|
|
||||||
3. ✅ Valid image model accepted (dall-e-3)
|
|
||||||
4. ✅ Invalid image model rejected (fake-dalle)
|
|
||||||
5. ✅ Inactive model rejected (gpt-image-1)
|
|
||||||
6. ✅ Valid size accepted (1024x1024 for dall-e-3)
|
|
||||||
7. ✅ Invalid size rejected (512x512 for dall-e-3)
|
|
||||||
8. ✅ All 5 active text models validate
|
|
||||||
9. ✅ All 2 active image models validate
|
|
||||||
|
|
||||||
**Result:** All validation logic working perfectly
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Test 4: Credit Calculation Integration
|
|
||||||
**Purpose:** Verify credit system integrates with AI costs
|
|
||||||
|
|
||||||
**Tests:**
|
|
||||||
1. ✅ Clustering credits: 2760 tokens → 19 credits
|
|
||||||
2. ✅ Profit margin: 99.7% (OpenAI cost $0.000523, Revenue $0.1900)
|
|
||||||
3. ✅ Minimum credits enforcement: 15 tokens → 10 credits (minimum)
|
|
||||||
4. ✅ High token count: 60,000 tokens → 600 credits
|
|
||||||
|
|
||||||
**Result:** Credit calculations work correctly with proper profit margins
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Test 5: REST API Serializer
|
|
||||||
**Purpose:** Verify API serialization works
|
|
||||||
|
|
||||||
**Tests:**
|
|
||||||
1. ✅ Single model serialization
|
|
||||||
2. ✅ Serialize all text models (5 models)
|
|
||||||
3. ✅ Serialize all image models (2 models)
|
|
||||||
4. ✅ Text model pricing fields (input_cost_per_1m, output_cost_per_1m)
|
|
||||||
5. ✅ Image model pricing fields (cost_per_image)
|
|
||||||
6. ✅ Image model sizes field (valid_sizes array)
|
|
||||||
7. ✅ Pricing display field
|
|
||||||
|
|
||||||
**Result:** All serialization working correctly with proper field names
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Test 6: End-to-End Integration
|
|
||||||
**Purpose:** Verify complete workflows work end-to-end
|
|
||||||
|
|
||||||
**Tests:**
|
|
||||||
1. ✅ Complete text generation workflow:
|
|
||||||
- Model validation
|
|
||||||
- OpenAI cost calculation ($0.000525)
|
|
||||||
- Credit calculation (20 credits)
|
|
||||||
- Revenue calculation ($0.2000)
|
|
||||||
- Profit margin (99.7%)
|
|
||||||
|
|
||||||
2. ✅ Complete image generation workflow:
|
|
||||||
- Model validation
|
|
||||||
- Size validation
|
|
||||||
- Cost calculation ($0.0400 per image)
|
|
||||||
|
|
||||||
3. ✅ All 7 active models verified (5 text + 2 image)
|
|
||||||
|
|
||||||
4. ✅ Database query performance for all models
|
|
||||||
|
|
||||||
**Result:** Complete workflows work perfectly from validation to cost calculation
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Features Verified
|
|
||||||
|
|
||||||
✅ Database-driven model pricing
|
|
||||||
✅ Cost calculation for text models (token-based)
|
|
||||||
✅ Cost calculation for image models (per-image)
|
|
||||||
✅ Model validation with active/inactive filtering
|
|
||||||
✅ Image size validation per model
|
|
||||||
✅ Credit calculation integration
|
|
||||||
✅ Profit margin calculation (99.7% for text, varies by model)
|
|
||||||
✅ REST API serialization
|
|
||||||
✅ Fallback to constants (safety mechanism)
|
|
||||||
✅ Django Admin interface with filters and bulk actions
|
|
||||||
✅ Lazy imports (circular dependency prevention)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Implementation Details
|
|
||||||
|
|
||||||
### Database Schema
|
|
||||||
- **Model:** `AIModelConfig`
|
|
||||||
- **Fields:** 15 (model_name, display_name, model_type, provider, costs, features, etc.)
|
|
||||||
- **Migration:** `0020_create_ai_model_config.py`
|
|
||||||
- **Seeded Models:** 9 (7 active, 2 inactive)
|
|
||||||
|
|
||||||
### Methods Implemented
|
|
||||||
```python
|
|
||||||
# Text model cost calculation
|
|
||||||
AIModelConfig.get_cost_for_tokens(input_tokens, output_tokens) -> Decimal
|
|
||||||
|
|
||||||
# Image model cost calculation
|
|
||||||
AIModelConfig.get_cost_for_images(num_images) -> Decimal
|
|
||||||
|
|
||||||
# Size validation
|
|
||||||
AIModelConfig.validate_size(size) -> bool
|
|
||||||
|
|
||||||
# Unified cost calculation (in ai_core.py)
|
|
||||||
AICore.calculate_cost(model, input_tokens, output_tokens, model_type) -> float
|
|
||||||
```
|
|
||||||
|
|
||||||
### Files Modified (7)
|
|
||||||
1. `billing/models.py` - AIModelConfig class (240 lines)
|
|
||||||
2. `billing/admin.py` - Admin interface with filters
|
|
||||||
3. `ai/ai_core.py` - 3 functions updated with database queries
|
|
||||||
4. `ai/validators.py` - 2 functions updated with database queries
|
|
||||||
5. `modules/billing/serializers.py` - AIModelConfigSerializer
|
|
||||||
6. `modules/billing/views.py` - AIModelConfigViewSet
|
|
||||||
7. `business/billing/urls.py` - API routing
|
|
||||||
|
|
||||||
### REST API Endpoints
|
|
||||||
- `GET /api/v1/billing/ai/models/` - List all active models
|
|
||||||
- `GET /api/v1/billing/ai/models/?model_type=text` - Filter by type
|
|
||||||
- `GET /api/v1/billing/ai/models/?provider=openai` - Filter by provider
|
|
||||||
- `GET /api/v1/billing/ai/models/<id>/` - Get specific model
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Cost Examples
|
|
||||||
|
|
||||||
### Text Generation (gpt-4o-mini)
|
|
||||||
- **OpenAI Cost:** 1000 input + 500 output tokens = $0.000450
|
|
||||||
- **Credits Charged:** 10 credits ($0.10)
|
|
||||||
- **Profit Margin:** 99.6%
|
|
||||||
|
|
||||||
### Image Generation (dall-e-3)
|
|
||||||
- **OpenAI Cost:** 1 image (1024x1024) = $0.0400
|
|
||||||
- **Credits:** Charged by customer configuration
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Fallback Safety Mechanism
|
|
||||||
|
|
||||||
All functions include try/except blocks that:
|
|
||||||
1. **Try:** Query database for model config
|
|
||||||
2. **Except:** Fall back to constants in `ai/constants.py`
|
|
||||||
3. **Result:** System never fails, always returns a valid cost
|
|
||||||
|
|
||||||
**Example:**
|
|
||||||
```python
|
|
||||||
try:
|
|
||||||
model_config = AIModelConfig.objects.get(model_name=model, is_active=True)
|
|
||||||
return model_config.get_cost_for_tokens(input, output)
|
|
||||||
except:
|
|
||||||
# Fallback to constants
|
|
||||||
rates = MODEL_RATES.get(model, {'input': 2.00, 'output': 8.00})
|
|
||||||
return calculate_with_rates(rates)
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Profit Margins
|
|
||||||
|
|
||||||
| Model | OpenAI Cost (1500 in + 500 out) | Credits | Revenue | Profit |
|
|
||||||
|-------|----------------------------------|---------|---------|--------|
|
|
||||||
| gpt-4o-mini | $0.000525 | 20 | $0.2000 | 99.7% |
|
|
||||||
| gpt-4o | $0.008750 | 20 | $0.2000 | 95.6% |
|
|
||||||
| gpt-4.1 | $0.007000 | 20 | $0.2000 | 96.5% |
|
|
||||||
| gpt-5.1 | $0.006875 | 20 | $0.2000 | 96.6% |
|
|
||||||
| gpt-5.2 | $0.009625 | 20 | $0.2000 | 95.2% |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Conclusion
|
|
||||||
|
|
||||||
✅ **SYSTEM IS 100% OPERATIONAL AND VERIFIED**
|
|
||||||
|
|
||||||
All 34 tests passed successfully. The AI Model Database Configuration system is:
|
|
||||||
- ✅ Fully functional
|
|
||||||
- ✅ Accurately calculating costs
|
|
||||||
- ✅ Properly validating models
|
|
||||||
- ✅ Successfully integrating with credit system
|
|
||||||
- ✅ Serving data via REST API
|
|
||||||
- ✅ Safe with fallback mechanisms
|
|
||||||
|
|
||||||
The system is ready for production use.
|
|
||||||
@@ -1,552 +0,0 @@
|
|||||||
# Architecture Knowledge Base
|
|
||||||
**Last Updated:** December 14, 2025
|
|
||||||
**Purpose:** Critical architectural patterns, common issues, and solutions reference
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔥 CRITICAL FIXES - December 2025
|
|
||||||
|
|
||||||
### PERMANENT FIX: Django Admin Custom Sidebar Not Showing on Subpages
|
|
||||||
**ROOT CAUSE**: Django's `ModelAdmin` view methods (`changelist_view`, `change_view`, etc.) do not call `AdminSite.each_context()`, so custom sidebar logic defined in `site.py` was bypassed on model list/detail/edit pages.
|
|
||||||
|
|
||||||
**SOLUTION IMPLEMENTED**:
|
|
||||||
1. ✅ Created `Igny8ModelAdmin` base class extending `UnfoldModelAdmin`
|
|
||||||
2. ✅ Overrides all view methods to inject `extra_context` with custom sidebar
|
|
||||||
3. ✅ Applied to 46+ admin classes across all modules
|
|
||||||
4. ✅ Sidebar now consistent on homepage, app index, and ALL model pages
|
|
||||||
|
|
||||||
**Files Modified**: `backend/igny8_core/admin/base.py`, all `*/admin.py` files
|
|
||||||
|
|
||||||
### PERMANENT FIX: User Swapping / Random Logout Issue
|
|
||||||
**ROOT CAUSE**: Django's database-backed sessions with in-memory user caching caused cross-request contamination at the process level.
|
|
||||||
|
|
||||||
**SOLUTION IMPLEMENTED**:
|
|
||||||
1. ✅ Redis-backed sessions (`SESSION_ENGINE = 'django.contrib.sessions.backends.cache'`)
|
|
||||||
2. ✅ Custom authentication backend without caching (`NoCacheModelBackend`)
|
|
||||||
3. ✅ Session integrity validation (stores and verifies account_id/user_id on every request)
|
|
||||||
4. ✅ Middleware never mutates `request.user` (uses Django's set value directly)
|
|
||||||
|
|
||||||
**See**: `CRITICAL-BUG-FIXES-DEC-2025.md` for complete details.
|
|
||||||
|
|
||||||
### PERMANENT FIX: useNavigate / useLocation Errors During HMR
|
|
||||||
**ROOT CAUSE**: Individual Suspense boundaries per route lost React Router context during Hot Module Replacement.
|
|
||||||
|
|
||||||
**SOLUTION IMPLEMENTED**:
|
|
||||||
1. ✅ Single top-level Suspense boundary around entire `<Routes>` component
|
|
||||||
2. ✅ Removed 100+ individual Suspense wrappers from route elements
|
|
||||||
3. ✅ Router context now persists through HMR automatically
|
|
||||||
|
|
||||||
**See**: `CRITICAL-BUG-FIXES-DEC-2025.md` for complete details.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Table of Contents
|
|
||||||
1. [Authentication & Session Management](#authentication--session-management)
|
|
||||||
2. [Site/Sector Architecture](#sitesector-architecture)
|
|
||||||
3. [State Management & Race Conditions](#state-management--race-conditions)
|
|
||||||
4. [Permission System](#permission-system)
|
|
||||||
5. [Frontend Component Dependencies](#frontend-component-dependencies)
|
|
||||||
6. [Common Pitfalls & Solutions](#common-pitfalls--solutions)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Authentication & Session Management
|
|
||||||
|
|
||||||
### Token Persistence Architecture
|
|
||||||
|
|
||||||
**Problem Pattern:**
|
|
||||||
- Zustand persist middleware writes to localStorage asynchronously
|
|
||||||
- API calls can happen before tokens are persisted
|
|
||||||
- Results in 403 "Authentication credentials were not provided" errors
|
|
||||||
|
|
||||||
**Solution Implemented:**
|
|
||||||
```typescript
|
|
||||||
// In authStore.ts login/register functions
|
|
||||||
// CRITICAL: Immediately persist tokens synchronously after setting state
|
|
||||||
const authState = {
|
|
||||||
state: { user, token, refreshToken, isAuthenticated: true },
|
|
||||||
version: 0
|
|
||||||
};
|
|
||||||
localStorage.setItem('auth-storage', JSON.stringify(authState));
|
|
||||||
```
|
|
||||||
|
|
||||||
**Key Principle:** Always write tokens to localStorage synchronously in auth actions, don't rely solely on persist middleware.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Logout & State Cleanup
|
|
||||||
|
|
||||||
**WRONG APPROACH (causes race conditions):**
|
|
||||||
```typescript
|
|
||||||
logout: () => {
|
|
||||||
localStorage.clear(); // ❌ BREAKS EVERYTHING
|
|
||||||
set({ user: null, token: null });
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**CORRECT APPROACH:**
|
|
||||||
```typescript
|
|
||||||
logout: () => {
|
|
||||||
// ✅ Selective removal - only auth-related keys
|
|
||||||
const authKeys = ['auth-storage', 'site-storage', 'sector-storage', 'billing-storage'];
|
|
||||||
authKeys.forEach(key => localStorage.removeItem(key));
|
|
||||||
|
|
||||||
// ✅ Reset dependent stores explicitly
|
|
||||||
useSiteStore.setState({ activeSite: null });
|
|
||||||
useSectorStore.setState({ activeSector: null, sectors: [] });
|
|
||||||
|
|
||||||
set({ user: null, token: null, isAuthenticated: false });
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Key Principle:** Never use `localStorage.clear()` - it breaks Zustand persist middleware initialization. Always selectively remove keys.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 403 Error Handling
|
|
||||||
|
|
||||||
**Problem Pattern:**
|
|
||||||
- 403 errors thrown before checking if it's an auth error
|
|
||||||
- Token validation code becomes unreachable
|
|
||||||
- Invalid tokens persist in localStorage
|
|
||||||
|
|
||||||
**WRONG ORDER:**
|
|
||||||
```typescript
|
|
||||||
// In api.ts
|
|
||||||
if (response.status === 403) {
|
|
||||||
throw new Error(response.statusText); // ❌ Thrown immediately
|
|
||||||
}
|
|
||||||
|
|
||||||
// This code NEVER runs (unreachable):
|
|
||||||
if (errorData?.detail?.includes('Authentication credentials')) {
|
|
||||||
logout(); // Never called!
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**CORRECT ORDER:**
|
|
||||||
```typescript
|
|
||||||
// Check for auth errors FIRST, then throw
|
|
||||||
if (response.status === 403) {
|
|
||||||
const errorData = JSON.parse(text);
|
|
||||||
|
|
||||||
// ✅ Check authentication BEFORE throwing
|
|
||||||
if (errorData?.detail?.includes('Authentication credentials')) {
|
|
||||||
const authState = useAuthStore.getState();
|
|
||||||
if (authState?.isAuthenticated) {
|
|
||||||
authState.logout();
|
|
||||||
window.location.href = '/signin';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Now throw the error
|
|
||||||
throw new Error(errorMessage);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Key Principle:** Handle authentication errors before throwing. Order matters in error handling logic.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Site/Sector Architecture
|
|
||||||
|
|
||||||
### Data Hierarchy
|
|
||||||
```
|
|
||||||
Account (Tenant)
|
|
||||||
└── Site (e.g., myblog.com)
|
|
||||||
└── Sector (e.g., Technology, Health)
|
|
||||||
└── Keywords
|
|
||||||
└── Clusters
|
|
||||||
└── Ideas
|
|
||||||
└── Content
|
|
||||||
```
|
|
||||||
|
|
||||||
### Where Sectors Are Used (Global Context)
|
|
||||||
|
|
||||||
**USES SECTORS (requires site/sector selection):**
|
|
||||||
- ✅ Planner Module (Keywords, Clusters, Ideas)
|
|
||||||
- ✅ Writer Module (Tasks, Content, Drafts, Published)
|
|
||||||
- ✅ Linker Module (Internal linking)
|
|
||||||
- ✅ Optimizer Module (Content optimization)
|
|
||||||
- ✅ Setup/Add Keywords page
|
|
||||||
- ✅ Seed Keywords reference data
|
|
||||||
|
|
||||||
**DOES NOT USE SECTORS (account-level only):**
|
|
||||||
- ❌ Billing/Plans pages (`/account/*`)
|
|
||||||
- ❌ Account Settings
|
|
||||||
- ❌ Team Management
|
|
||||||
- ❌ User Profile
|
|
||||||
- ❌ Admin Dashboard
|
|
||||||
- ❌ System Settings
|
|
||||||
|
|
||||||
### Sector Loading Pattern
|
|
||||||
|
|
||||||
**Architecture Decision:**
|
|
||||||
- Sectors loaded by **PageHeader component** (not AppLayout)
|
|
||||||
- Only loads when `hideSiteSector={false}` prop is set
|
|
||||||
- Account/billing pages pass `hideSiteSector={true}` to skip loading
|
|
||||||
|
|
||||||
**Implementation:**
|
|
||||||
```typescript
|
|
||||||
// PageHeader.tsx
|
|
||||||
useEffect(() => {
|
|
||||||
if (hideSiteSector) return; // Skip for account pages
|
|
||||||
|
|
||||||
const currentSiteId = activeSite?.id ?? null;
|
|
||||||
if (currentSiteId && activeSite?.is_active) {
|
|
||||||
loadSectorsForSite(currentSiteId);
|
|
||||||
}
|
|
||||||
}, [activeSite?.id, hideSiteSector]);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Key Principle:** Lazy-load sectors only when components need them. Don't load globally for all pages.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Site/Sector Store Persistence
|
|
||||||
|
|
||||||
**Storage Keys:**
|
|
||||||
- `site-storage` - Active site selection
|
|
||||||
- `sector-storage` - Active sector selection
|
|
||||||
|
|
||||||
**Reset Pattern:**
|
|
||||||
```typescript
|
|
||||||
// When site changes, reset sector if it belongs to different site
|
|
||||||
if (currentSector && currentSector.site_id !== newSiteId) {
|
|
||||||
set({ activeSector: null });
|
|
||||||
localStorage.setItem('sector-storage', JSON.stringify({
|
|
||||||
state: { activeSector: null },
|
|
||||||
version: 0
|
|
||||||
}));
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Key Principle:** Sector selection is site-scoped. Always validate sector belongs to active site.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## State Management & Race Conditions
|
|
||||||
|
|
||||||
### Common Race Condition Patterns
|
|
||||||
|
|
||||||
#### 1. User Switching
|
|
||||||
**Problem:** Rapid logout → login leaves stale state in stores
|
|
||||||
|
|
||||||
**Solution:**
|
|
||||||
```typescript
|
|
||||||
logout: () => {
|
|
||||||
// Reset ALL dependent stores explicitly
|
|
||||||
import('./siteStore').then(({ useSiteStore }) => {
|
|
||||||
useSiteStore.setState({ activeSite: null, loading: false, error: null });
|
|
||||||
});
|
|
||||||
import('./sectorStore').then(({ useSectorStore }) => {
|
|
||||||
useSectorStore.setState({ activeSector: null, sectors: [], loading: false, error: null });
|
|
||||||
});
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 2. API Calls Before Token Persistence
|
|
||||||
**Problem:** API calls happen before Zustand persist writes token
|
|
||||||
|
|
||||||
**Solution:** Synchronous localStorage write immediately after state update (see Authentication section)
|
|
||||||
|
|
||||||
#### 3. Module Loading Failures
|
|
||||||
**Problem:** 404 errors during page navigation cause module loading to fail
|
|
||||||
|
|
||||||
**Solution:** Ensure API endpoints exist before pages try to load them. Use conditional rendering based on route.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Zustand Persist Middleware Gotchas
|
|
||||||
|
|
||||||
**Issue 1: Version Mismatch**
|
|
||||||
```typescript
|
|
||||||
// Stored format
|
|
||||||
{ state: { user, token }, version: 0 }
|
|
||||||
|
|
||||||
// If version changes, persist middleware clears state
|
|
||||||
```
|
|
||||||
|
|
||||||
**Issue 2: Async Hydration**
|
|
||||||
- State rehydration from localStorage is async
|
|
||||||
- Can cause brief flash of "no user" state
|
|
||||||
|
|
||||||
**Solution:** Use loading states or check both store AND localStorage:
|
|
||||||
```typescript
|
|
||||||
const getAuthToken = (): string | null => {
|
|
||||||
// Try Zustand store first
|
|
||||||
const authState = useAuthStore.getState();
|
|
||||||
if (authState?.token) return authState.token;
|
|
||||||
|
|
||||||
// Fallback to localStorage
|
|
||||||
const stored = localStorage.getItem('auth-storage');
|
|
||||||
return JSON.parse(stored)?.state?.token || null;
|
|
||||||
};
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Permission System
|
|
||||||
|
|
||||||
### Superuser/Developer Bypass Pattern
|
|
||||||
|
|
||||||
**Critical Locations for Bypass:**
|
|
||||||
1. Middleware - `auth/middleware.py`
|
|
||||||
2. Permission Classes - `api/permissions.py`
|
|
||||||
3. ViewSet Querysets - `api/base.py`
|
|
||||||
4. Validation Functions - `auth/utils.py`
|
|
||||||
|
|
||||||
**Standard Bypass Check:**
|
|
||||||
```python
|
|
||||||
def check_bypass(user):
|
|
||||||
return (
|
|
||||||
user.is_superuser or
|
|
||||||
user.role == 'developer' or
|
|
||||||
is_system_account_user(user)
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Apply at ALL levels:**
|
|
||||||
- Middleware request validation
|
|
||||||
- DRF permission `has_permission()`
|
|
||||||
- ViewSet `get_queryset()` filtering
|
|
||||||
- Custom validation functions
|
|
||||||
|
|
||||||
**Key Principle:** Bypass checks must be consistent across all permission layers. Missing one layer breaks superuser access.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### System Account Pattern
|
|
||||||
|
|
||||||
**Reserved Accounts:**
|
|
||||||
- `aws-admin` - System automation account
|
|
||||||
- `default-account` - Default tenant fallback
|
|
||||||
|
|
||||||
**Check Function:**
|
|
||||||
```python
|
|
||||||
def is_system_account_user(user):
|
|
||||||
if not user or not user.account:
|
|
||||||
return False
|
|
||||||
return user.account.slug in ['aws-admin', 'default-account']
|
|
||||||
```
|
|
||||||
|
|
||||||
**Usage:** Always include in bypass checks alongside superuser/developer.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Frontend Component Dependencies
|
|
||||||
|
|
||||||
### PageHeader Component
|
|
||||||
**Dependencies:**
|
|
||||||
- `useSiteStore` - Active site
|
|
||||||
- `useSectorStore` - Active sector
|
|
||||||
- `SiteAndSectorSelector` - Dropdown component
|
|
||||||
|
|
||||||
**Props:**
|
|
||||||
- `hideSiteSector: boolean` - Skip site/sector display and loading
|
|
||||||
- `title: string` - Page title
|
|
||||||
- `navigation: ReactNode` - Optional module tabs
|
|
||||||
|
|
||||||
**Used By:**
|
|
||||||
- All Planner pages
|
|
||||||
- All Writer pages
|
|
||||||
- All Optimizer pages
|
|
||||||
- Setup pages
|
|
||||||
- Seed Keywords page
|
|
||||||
|
|
||||||
**NOT Used By:**
|
|
||||||
- Account/billing pages (use plain headers instead)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Module Navigation Pattern
|
|
||||||
|
|
||||||
**Component:** `ModuleNavigationTabs.tsx`
|
|
||||||
|
|
||||||
**CRITICAL:** Must be wrapped in `<Router>` context
|
|
||||||
- Uses `useLocation()` and `useNavigate()` hooks
|
|
||||||
- Cannot be used outside `<Routes>` tree
|
|
||||||
|
|
||||||
**Common Error:**
|
|
||||||
```
|
|
||||||
Error: useLocation() may be used only in the context of a <Router> component
|
|
||||||
```
|
|
||||||
|
|
||||||
**Cause:** Component rendered outside React Router context
|
|
||||||
|
|
||||||
**Solution:** Ensure component is within `<Route>` element in App.tsx
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Common Pitfalls & Solutions
|
|
||||||
|
|
||||||
### Pitfall 1: Frontend 403 Errors After User Switch
|
|
||||||
|
|
||||||
**Symptoms:**
|
|
||||||
- "Authentication credentials were not provided"
|
|
||||||
- User appears logged in but API calls fail
|
|
||||||
- Manually clearing cache fixes it
|
|
||||||
|
|
||||||
**Root Cause:** Invalid tokens persisting in localStorage after logout
|
|
||||||
|
|
||||||
**Solution:**
|
|
||||||
1. Check 403 handler runs BEFORE throwing error
|
|
||||||
2. Ensure logout clears specific auth keys (not `localStorage.clear()`)
|
|
||||||
3. Add immediate token persistence after login
|
|
||||||
|
|
||||||
**Prevention:** See "Authentication & Session Management" section
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Pitfall 2: Sector 404 Errors on Billing Pages
|
|
||||||
|
|
||||||
**Symptoms:**
|
|
||||||
- `GET /v1/auth/sites/{id}/sectors/` returns 404
|
|
||||||
- "Failed to fetch dynamically imported module" error
|
|
||||||
- Billing pages don't load
|
|
||||||
|
|
||||||
**Root Cause:** AppLayout loading sectors for ALL pages globally
|
|
||||||
|
|
||||||
**Solution:** Move sector loading to PageHeader component (lazy loading)
|
|
||||||
|
|
||||||
**Prevention:** Only load data when components that need it are mounted
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Pitfall 3: Module Loading Failures After Git Commits
|
|
||||||
|
|
||||||
**Symptoms:**
|
|
||||||
- React Router context errors
|
|
||||||
- "useLocation() may be used only in context of <Router>" errors
|
|
||||||
- Pages work after rebuild but fail after git push
|
|
||||||
|
|
||||||
**Root Cause:** Docker build cache not invalidated properly
|
|
||||||
|
|
||||||
**Solution:**
|
|
||||||
```bash
|
|
||||||
# Force clean rebuild
|
|
||||||
docker compose -f docker-compose.app.yml down
|
|
||||||
docker compose -f docker-compose.app.yml build --no-cache igny8_frontend
|
|
||||||
docker compose -f docker-compose.app.yml up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
**Prevention:** Use `--no-cache` flag when rebuilding after major changes
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Pitfall 4: Plan Selection Issues in Pricing Page
|
|
||||||
|
|
||||||
**Symptoms:**
|
|
||||||
- Monthly/Annual toggle missing
|
|
||||||
- Pre-selected plan not highlighted
|
|
||||||
- Discount calculation wrong
|
|
||||||
|
|
||||||
**Root Cause:**
|
|
||||||
1. PricingTable component missing `showToggle` prop
|
|
||||||
2. Backend missing `is_featured` and `annual_discount_percent` fields
|
|
||||||
3. Frontend not calculating annual price from discount
|
|
||||||
|
|
||||||
**Solution:**
|
|
||||||
1. Add fields to Plan model with migration
|
|
||||||
2. Pass `annualDiscountPercent` to PricingTable
|
|
||||||
3. Calculate: `annualPrice = monthlyPrice * 12 * (1 - discount/100)`
|
|
||||||
|
|
||||||
**Files Modified:**
|
|
||||||
- `backend/igny8_core/auth/models.py`
|
|
||||||
- `backend/igny8_core/auth/serializers.py`
|
|
||||||
- `frontend/src/services/billing.api.ts`
|
|
||||||
- `frontend/src/components/ui/pricing-table/PricingTable.tsx`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Pitfall 5: Adjacent JSX Elements Error
|
|
||||||
|
|
||||||
**Symptoms:**
|
|
||||||
- "Adjacent JSX elements must be wrapped in an enclosing tag"
|
|
||||||
- Build fails but line numbers don't help
|
|
||||||
|
|
||||||
**Root Cause:** Mismatched opening/closing tags (usually missing `</div>`)
|
|
||||||
|
|
||||||
**Debugging Strategy:**
|
|
||||||
1. Use TypeScript compiler: `npx tsc --noEmit <file>`
|
|
||||||
2. Count opening vs closing tags: `grep -c "<div" vs grep -c "</div>"`
|
|
||||||
3. Check conditionals have matching closing parens/braces
|
|
||||||
|
|
||||||
**Common Pattern:**
|
|
||||||
```tsx
|
|
||||||
{condition && (
|
|
||||||
<div>
|
|
||||||
{/* Content */}
|
|
||||||
</div>
|
|
||||||
{/* Missing closing parenthesis causes "adjacent elements" error */}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Solution:** Ensure every opening bracket has matching close bracket
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Best Practices Summary
|
|
||||||
|
|
||||||
### State Management
|
|
||||||
✅ **DO:** Immediately persist auth tokens synchronously
|
|
||||||
✅ **DO:** Selectively remove localStorage keys
|
|
||||||
✅ **DO:** Reset dependent stores on logout
|
|
||||||
❌ **DON'T:** Use `localStorage.clear()`
|
|
||||||
❌ **DON'T:** Rely solely on Zustand persist middleware timing
|
|
||||||
|
|
||||||
### Error Handling
|
|
||||||
✅ **DO:** Check authentication errors BEFORE throwing
|
|
||||||
✅ **DO:** Force logout on invalid tokens
|
|
||||||
✅ **DO:** Redirect to login after logout
|
|
||||||
❌ **DON'T:** Throw errors before checking auth status
|
|
||||||
❌ **DON'T:** Leave invalid tokens in storage
|
|
||||||
|
|
||||||
### Component Architecture
|
|
||||||
✅ **DO:** Lazy-load data at component level
|
|
||||||
✅ **DO:** Skip unnecessary data loading (hideSiteSector pattern)
|
|
||||||
✅ **DO:** Keep components in Router context
|
|
||||||
❌ **DON'T:** Load data globally in AppLayout
|
|
||||||
❌ **DON'T:** Use Router hooks outside Router context
|
|
||||||
|
|
||||||
### Permission System
|
|
||||||
✅ **DO:** Implement bypass at ALL permission layers
|
|
||||||
✅ **DO:** Include system accounts in bypass checks
|
|
||||||
✅ **DO:** Use consistent bypass logic everywhere
|
|
||||||
❌ **DON'T:** Forget middleware layer bypass
|
|
||||||
❌ **DON'T:** Mix permission approaches
|
|
||||||
|
|
||||||
### Docker Builds
|
|
||||||
✅ **DO:** Use `--no-cache` after major changes
|
|
||||||
✅ **DO:** Restart containers after rebuilds
|
|
||||||
✅ **DO:** Check logs for module loading errors
|
|
||||||
❌ **DON'T:** Trust build cache after git commits
|
|
||||||
❌ **DON'T:** Deploy without testing fresh build
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Quick Reference: File Locations
|
|
||||||
|
|
||||||
### Authentication
|
|
||||||
- Token handling: `frontend/src/services/api.ts`
|
|
||||||
- Auth store: `frontend/src/store/authStore.ts`
|
|
||||||
- Middleware: `backend/igny8_core/auth/middleware.py`
|
|
||||||
|
|
||||||
### Permissions
|
|
||||||
- Permission classes: `backend/igny8_core/api/permissions.py`
|
|
||||||
- Base viewsets: `backend/igny8_core/api/base.py`
|
|
||||||
- Validation utils: `backend/igny8_core/auth/utils.py`
|
|
||||||
|
|
||||||
### Site/Sector
|
|
||||||
- Site store: `frontend/src/store/siteStore.ts`
|
|
||||||
- Sector store: `frontend/src/store/sectorStore.ts`
|
|
||||||
- PageHeader: `frontend/src/components/common/PageHeader.tsx`
|
|
||||||
|
|
||||||
### Billing
|
|
||||||
- Billing API: `frontend/src/services/billing.api.ts`
|
|
||||||
- Plans page: `frontend/src/pages/account/PlansAndBillingPage.tsx`
|
|
||||||
- Plan model: `backend/igny8_core/auth/models.py`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**End of Knowledge Base**
|
|
||||||
*Update this document when architectural patterns change or new common issues are discovered.*
|
|
||||||
664
CHANGELOG.md
664
CHANGELOG.md
@@ -1,7 +1,20 @@
|
|||||||
# IGNY8 Change Log
|
# IGNY8 Change Log
|
||||||
|
|
||||||
**Current Version:** v1.0.5
|
**Current Version:** 1.0.5
|
||||||
**Last Updated:** December 12, 2025
|
**Last Updated:** December 25, 2025
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Version History
|
||||||
|
|
||||||
|
| Version | Date | Summary |
|
||||||
|
|---------|------|---------|
|
||||||
|
| 1.0.5 | Dec 12, 2025 | Purchase Credits tab, UI reorganization |
|
||||||
|
| 1.0.4 | Dec 12, 2025 | Credit Activity tab, Cost Reference panel |
|
||||||
|
| 1.0.3 | Dec 12, 2025 | Usage Limits color variety, Cost Breakdown |
|
||||||
|
| 1.0.2 | Dec 12, 2025 | Usage Analytics consolidation, Design system |
|
||||||
|
| 1.0.1 | Dec 12, 2025 | Usage summary endpoint, Plan limits UI |
|
||||||
|
| 1.0.0 | Dec 12, 2025 | Initial production release |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -10,683 +23,140 @@
|
|||||||
### Added
|
### Added
|
||||||
- **Purchase Credits Tab**: New dedicated tab in Plans & Billing
|
- **Purchase Credits Tab**: New dedicated tab in Plans & Billing
|
||||||
- Separated credit package purchases from Credits Overview
|
- Separated credit package purchases from Credits Overview
|
||||||
- Cleaner organization: Current Plan → Credits Overview → Purchase Credits → Billing History
|
|
||||||
- Shows all available credit packages in horizontal scrollable layout
|
- Shows all available credit packages in horizontal scrollable layout
|
||||||
- Payment method requirement notice for users without payment methods
|
- Payment method requirement notice for users without payment methods
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
- **Plans & Billing Tab Organization**:
|
- **Plans & Billing Tab Organization**: 4 tabs instead of 3
|
||||||
- **Before:** 3 tabs (Current Plan, Credits Overview, Billing History)
|
- Current Plan → Credits Overview → Purchase Credits → Billing History
|
||||||
- **After:** 4 tabs (Current Plan, Credits Overview, Purchase Credits, Billing History)
|
- **Link Updates**: Purchase Credits buttons now link to correct tab
|
||||||
- Credits Overview now focused on balance, usage, and analytics
|
|
||||||
- Purchase Credits dedicated to credit packages and purchasing
|
|
||||||
|
|
||||||
- **Link Updates**:
|
|
||||||
- "Purchase Credits" button in Current Plan tab → Links to Purchase Credits tab
|
|
||||||
- Upgrade CTA in Usage Limits Panel → Links to Purchase Credits tab with `?tab=purchase` query param
|
|
||||||
|
|
||||||
### Technical
|
|
||||||
- Updated `PlansAndBillingPage.tsx` tab type to include 'purchase'
|
|
||||||
- Moved credit packages section from Credits tab to new Purchase tab
|
|
||||||
- Updated navigation links to point to correct tab
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## v1.0.4 - December 12, 2025
|
## v1.0.4 - December 12, 2025
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
- **Credit Activity Tab**: New dedicated tab in Usage Analytics for transaction history
|
- **Credit Activity Tab**: Transaction history in Usage Analytics
|
||||||
- Moved credit transaction table from Limits & Usage to separate Activity tab
|
- **Credit Costs Reference Panel**: Shows cost per operation type
|
||||||
- Better organization: Limits → Activity → API Usage
|
|
||||||
- Full transaction history with pagination
|
|
||||||
|
|
||||||
- **Credit Costs Reference Panel**: New component in Plans & Billing → Credits Overview
|
|
||||||
- Shows credit cost for each operation type (Clustering, Idea Generation, etc.)
|
|
||||||
- Clean card layout with IGNY8 brand colors
|
|
||||||
- Proper hover states and border colors using CSS variables
|
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
- **Usage Analytics Reorganization**:
|
- **Usage Analytics Reorganization**: 3 tabs (Limits & Usage, Activity, API Usage)
|
||||||
- **Before:** 2 tabs (Limits & Usage, API Usage)
|
|
||||||
- **After:** 3 tabs (Limits & Usage, Activity, API Usage)
|
|
||||||
- Limits tab now shows only plan limits (cleaner, focused)
|
|
||||||
- Activity tab dedicated to credit transaction history
|
|
||||||
|
|
||||||
- **Plans & Billing Credits Tab Enhancement**:
|
|
||||||
- Now includes "Credit Costs per Operation" reference panel
|
|
||||||
- Shows both cost analytics AND cost reference in same location
|
|
||||||
- Users can see historical costs and reference costs side-by-side
|
|
||||||
|
|
||||||
### Fixed
|
|
||||||
- **Color Scheme Consistency**: All credit cost displays now use proper IGNY8 brand colors
|
|
||||||
- Badge colors: `var(--color-brand-*)` instead of generic blue
|
|
||||||
- Hover states: `var(--color-brand-300)` for light mode, `var(--color-brand-600)` for dark mode
|
|
||||||
- Removed hardcoded blue colors from billing components
|
|
||||||
|
|
||||||
### Technical
|
|
||||||
- Created `CreditCostsPanel.tsx` - Standalone reference panel for operation costs
|
|
||||||
- Enhanced `BillingUsagePanel.tsx` with `showOnlyActivity` prop for flexible rendering
|
|
||||||
- Updated `UsageAnalyticsPage.tsx` to support 3-tab layout
|
|
||||||
- Updated `PlansAndBillingPage.tsx` to include both cost breakdown and cost reference
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## v1.0.3 - December 12, 2025
|
## v1.0.3 - December 12, 2025
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
- **Color Variety in Usage Limits**: Introduced subtle color accents for different limit types to improve visual distinction
|
- **Color Variety in Usage Limits**: Subtle color accents per limit type
|
||||||
- Hard limits: Sites (green), Users (cyan), Keywords (purple), Clusters (orange)
|
- **Credit Cost Breakdown Panel**: Detailed cost analytics
|
||||||
- Monthly limits: Content Ideas (brand blue), Content Words (indigo), Basic Images (teal), Premium Images (cyan), Image Prompts (pink)
|
|
||||||
- Each limit card now displays with its own themed color for icon background and progress bar
|
|
||||||
|
|
||||||
- **Credit Cost Breakdown Panel**: New component showing detailed cost analytics
|
|
||||||
- Cost per operation type with color-coded cards
|
|
||||||
- Total cost, average daily cost, and cost per credit summary
|
|
||||||
- Operation count and average credits per operation
|
|
||||||
- Moved from Usage Analytics to Plans & Billing → Credits Overview tab
|
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
- **Plans & Billing Page Enhancement**: Credit cost analytics now in Credits Overview tab
|
- **Plans & Billing Enhancement**: Cost analytics in Credits Overview tab
|
||||||
- Better organization with cost data alongside credit balance information
|
- **Comprehensive Color Refactoring**: All components use CSS variables
|
||||||
- Improved user flow: Plan → Credits + Costs → Invoices
|
|
||||||
|
|
||||||
- **Usage Analytics Simplification**: Removed Cost Breakdown tab
|
|
||||||
- Reduced from 3 tabs to 2 tabs (Limits & Usage, API Usage)
|
|
||||||
- Cost information now in more logical location (Plans & Billing)
|
|
||||||
|
|
||||||
- **Comprehensive Color Refactoring**: Replaced ALL remaining hardcoded blue/purple colors
|
|
||||||
- Fixed `PurchaseCreditsPage` payment instruction box (blue-50 → info-50)
|
|
||||||
- Fixed `PlansAndBillingPage` pending payment notice (blue-50 → info-50)
|
|
||||||
- Fixed plan change policy card (blue-50 → brand-50)
|
|
||||||
- Ensured all components use CSS variables from design system
|
|
||||||
- No more hardcoded hex colors (#3b82f6, #2563eb) anywhere in account pages
|
|
||||||
|
|
||||||
### Improved
|
|
||||||
- **Visual Hierarchy**: Multi-color approach prevents "mono-color fatigue"
|
|
||||||
- Different sections now visually distinct at a glance
|
|
||||||
- Color-coding aids in quick identification of limit types
|
|
||||||
- Maintains brand consistency while adding subtle variety
|
|
||||||
|
|
||||||
- **Design System Compliance**: 100% adherence to IGNY8 color tokens
|
|
||||||
- All colors reference `var(--color-*)` CSS variables
|
|
||||||
- Easy theme customization through token updates
|
|
||||||
- Dark mode colors automatically derived from tokens
|
|
||||||
|
|
||||||
### Technical
|
|
||||||
- Created `CreditCostBreakdownPanel.tsx` component
|
|
||||||
- Updated `UsageLimitsPanel.tsx` with color configuration system
|
|
||||||
- Enhanced `LimitCard` component with `accentColor` prop
|
|
||||||
- Removed Cost Breakdown tab from `UsageAnalyticsPage.tsx`
|
|
||||||
- Added cost breakdown section to `PlansAndBillingPage.tsx` Credits tab
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## v1.0.2 - December 12, 2025
|
## v1.0.2 - December 12, 2025
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
- **UI/UX Reorganization**: Consolidated Usage Analytics page layout
|
- **UI/UX Reorganization**: Consolidated Usage Analytics layout
|
||||||
- Renamed "Plan Limits" tab to "Limits & Usage" for better clarity
|
- **Design System Enforcement**: Standardized IGNY8 brand colors
|
||||||
- Merged credit usage content into Limits & Usage tab (previously split across multiple tabs)
|
|
||||||
- Removed redundant "Credit Balance" tab
|
|
||||||
- Reduced tab count from 5 to 3 (Limits & Usage, API Usage, Cost Breakdown)
|
|
||||||
|
|
||||||
- **Design System Enforcement**: Standardized IGNY8 brand colors across all account pages
|
|
||||||
- Replaced all hardcoded blue colors (#3b82f6, #2563eb, bg-blue-500/600/700) with CSS variable `var(--color-brand-500)`
|
|
||||||
- Removed parallel `account-colors.css` file with custom color definitions
|
|
||||||
- Updated all account pages to use centralized color system from `index.css`
|
|
||||||
- Applied consistent brand color (#4F46E5) to: AccountSettingsPage, TeamManagementPage, PlansAndBillingPage, PurchaseCreditsPage, UsageAnalyticsPage
|
|
||||||
- Updated component colors: UsageLimitsPanel, BillingUsagePanel progress bars, buttons, badges, borders, focus states
|
|
||||||
- Enhanced visual consistency across dashboard, loaders, form inputs, selection states, hover effects
|
|
||||||
|
|
||||||
### Removed
|
### Removed
|
||||||
- `frontend/src/styles/account-colors.css` - deprecated custom color file
|
- `frontend/src/styles/account-colors.css` - deprecated custom color file
|
||||||
- Import of account-colors.css from index.css
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## v1.0.1 - December 12, 2025
|
## v1.0.1 - December 12, 2025
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
- Usage summary endpoint routing - added direct URL path for `/api/v1/billing/usage-summary/` as standalone user-facing endpoint (separated from admin-only `BillingViewSet`)
|
- Usage summary endpoint routing
|
||||||
- Endpoint accessibility issue - properly registered `get_usage_summary` function view in billing URLs
|
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
- **Frontend Plan Limits Display**: Complete UI integration for usage tracking and limits
|
- **Frontend Plan Limits Display**: Visual progress bars for all plan limits
|
||||||
- `UsageLimitsPanel` component - visual progress bars for all plan limits with color-coded status (blue/yellow/red)
|
- **IGNY8 Brand Styling**: Custom CSS design system
|
||||||
- Plan Limits tab in Usage Analytics page (now default/first tab)
|
|
||||||
- Limit display in pricing table comparison with "Unlimited" formatting for high values
|
|
||||||
- Days until reset counter for monthly limits
|
|
||||||
- Upgrade CTA when approaching limits
|
|
||||||
|
|
||||||
- **IGNY8 Brand Styling**: Custom CSS design system for account section (`account-colors.css`)
|
|
||||||
- Brand gradients (primary blue, success green, warning amber, danger red, purple, teal)
|
|
||||||
- Enhanced card variants with top color bars
|
|
||||||
- Animated progress bars with shimmer effects
|
|
||||||
- Stat numbers with gradient text effects
|
|
||||||
- Custom badges and plan cards
|
|
||||||
- Billing tables with hover effects
|
|
||||||
- Dark mode support throughout
|
|
||||||
|
|
||||||
### Changed
|
|
||||||
- Usage Analytics page tab order - Plan Limits now first tab, followed by Credit Usage, Credit Balance, API Usage, Cost Breakdown
|
|
||||||
- Pricing table enhanced to display all plan limits (sites, users, words/month, ideas/month, images/month, credits/month)
|
|
||||||
- Plans and Billing page now passes complete limit data to pricing table component
|
|
||||||
|
|
||||||
### Documentation
|
|
||||||
- Added API endpoint rules to `.cursorrules` - enforce using existing `/api/v1/` structure, test via `/api/docs/` Swagger, no parallel API systems
|
|
||||||
- Updated `PLAN-LIMITS.md` with correct endpoint path documentation
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## v1.0.0 - December 12, 2025 (Production Release)
|
## v1.0.0 - December 12, 2025 (Initial Production Release)
|
||||||
|
|
||||||
### 🎉 INITIAL PRODUCTION RELEASE
|
### Core Features
|
||||||
|
|
||||||
This marks the first production-ready release of IGNY8, featuring a complete multi-tenant AI-powered content platform.
|
|
||||||
|
|
||||||
### 🆕 Latest Additions (v1.0.0 Final)
|
|
||||||
|
|
||||||
**Plan Limits System** - Implemented comprehensive usage limits tied to subscription plans:
|
|
||||||
- **Hard Limits** (persistent, never reset):
|
|
||||||
- `max_sites`: Maximum number of sites per account
|
|
||||||
- `max_users`: Maximum team members
|
|
||||||
- `max_keywords`: Total keywords allowed
|
|
||||||
- `max_clusters`: Total clusters allowed
|
|
||||||
|
|
||||||
- **Monthly Limits** (reset on billing cycle):
|
|
||||||
- `max_content_ideas`: New content ideas per month
|
|
||||||
- `max_content_words`: Words generated per month
|
|
||||||
- `max_images_basic`: Basic AI images per month
|
|
||||||
- `max_images_premium`: Premium AI images per month
|
|
||||||
- `max_image_prompts`: Image prompts per month
|
|
||||||
|
|
||||||
- **Enforcement Features**:
|
|
||||||
- Pre-operation validation (blocks exceeding actions)
|
|
||||||
- Real-time usage tracking and incrementation
|
|
||||||
- Automatic monthly reset via Celery Beat task
|
|
||||||
- Usage summary API endpoint (`/api/v1/billing/usage-summary/`)
|
|
||||||
- Warning notifications at 80% threshold
|
|
||||||
- Clear error messages with upgrade prompts
|
|
||||||
|
|
||||||
- **Technical Implementation**:
|
|
||||||
- `PlanLimitUsage` model tracks monthly consumption
|
|
||||||
- `LimitService` handles all check/increment/reset operations
|
|
||||||
- `word_counter.py` utility for accurate HTML word counting
|
|
||||||
- Integrated into Site creation, content generation, image generation
|
|
||||||
- Celery scheduled tasks: `reset_monthly_plan_limits` (daily), `check_approaching_limits` (daily)
|
|
||||||
|
|
||||||
|
|
||||||
### ✨ Core Features Implemented
|
|
||||||
|
|
||||||
**Multi-Tenancy System:**
|
**Multi-Tenancy System:**
|
||||||
- Complete account isolation with row-level security
|
- Complete account isolation with row-level security
|
||||||
- Multi-site management per account
|
- Multi-site management per account
|
||||||
- Sector-based content organization
|
- Sector-based content organization
|
||||||
- Role-based access control (Owner, Admin, Editor, Viewer)
|
- Role-based access control (Owner, Admin, Editor, Viewer)
|
||||||
- Session management with Redis backend
|
|
||||||
- JWT authentication with refresh tokens
|
|
||||||
|
|
||||||
**Planner Module:**
|
**Planner Module:**
|
||||||
- Bulk keyword import (CSV, unlimited rows)
|
- Bulk keyword import (CSV)
|
||||||
- AI-powered keyword clustering (GPT-4)
|
- AI-powered keyword clustering (GPT-4)
|
||||||
- Global seed keyword database (50+ industries)
|
- Global seed keyword database (50+ industries)
|
||||||
- Content idea generation from clusters
|
- Content idea generation from clusters
|
||||||
- Search intent classification
|
|
||||||
- Advanced filtering and search
|
|
||||||
|
|
||||||
**Writer Module:**
|
**Writer Module:**
|
||||||
- AI content generation (GPT-4, customizable word counts)
|
- AI content generation (GPT-4)
|
||||||
- Task management and queuing
|
- Task management and queuing
|
||||||
- Content taxonomy system (categories, tags)
|
- Content taxonomy system
|
||||||
- HTML content editor integration
|
|
||||||
- SEO metadata management
|
|
||||||
- Status workflow (draft, review, published)
|
- Status workflow (draft, review, published)
|
||||||
|
|
||||||
**Image Generation:**
|
**Image Generation:**
|
||||||
- Dual AI providers (DALL-E 3, Runware)
|
- DALL-E 3 and Runware providers
|
||||||
- Featured and in-article images
|
- Featured and in-article images
|
||||||
- Smart prompt extraction from content
|
|
||||||
- Multiple image sizes and formats
|
|
||||||
- Automatic alt text generation
|
- Automatic alt text generation
|
||||||
- Batch image processing
|
|
||||||
|
|
||||||
**Automation Pipeline:**
|
**Automation Pipeline:**
|
||||||
- 7-stage automation (keywords → clusters → ideas → tasks → content → prompts → images)
|
- 7-stage automation (keywords → images)
|
||||||
- Scheduled runs (daily, weekly, monthly)
|
- Scheduled runs (daily, weekly, monthly)
|
||||||
- Configurable batch sizes and delays
|
- Configurable batch sizes
|
||||||
- Credit estimation and buffering
|
|
||||||
- Pause/resume functionality
|
|
||||||
- Detailed activity logging
|
|
||||||
|
|
||||||
**Internal Linking:**
|
|
||||||
- AI-powered link candidate discovery
|
|
||||||
- Relevance scoring and smart injection
|
|
||||||
- Context-aware anchor text
|
|
||||||
- Bi-directional linking
|
|
||||||
- Link density control
|
|
||||||
|
|
||||||
**Content Optimization:**
|
|
||||||
- 15+ factor SEO analysis
|
|
||||||
- Readability scoring
|
|
||||||
- AI-powered content enhancement
|
|
||||||
- Before/after comparison
|
|
||||||
- Batch optimization support
|
|
||||||
|
|
||||||
**WordPress Integration:**
|
**WordPress Integration:**
|
||||||
- One-click publishing to WordPress
|
- One-click publishing
|
||||||
- Bidirectional status sync
|
- Bidirectional sync
|
||||||
- Taxonomy synchronization
|
- Taxonomy synchronization
|
||||||
- Featured image upload
|
|
||||||
- SEO metadata sync
|
|
||||||
- Multiple site support
|
|
||||||
- API key authentication
|
|
||||||
|
|
||||||
**Billing & Credits:**
|
**Billing & Credits:**
|
||||||
- Usage-based credit system
|
- Usage-based credit system
|
||||||
- Plan-based hard limits (sites, users, keywords, clusters)
|
- Plan-based limits
|
||||||
- Monthly usage limits (content ideas, words, images, prompts)
|
- Monthly usage tracking
|
||||||
- Automatic limit enforcement with clear error messages
|
|
||||||
- Real-time usage tracking and reporting
|
|
||||||
- Monthly limit reset on billing cycle
|
|
||||||
- Usage warning notifications
|
|
||||||
- Multiple payment methods (Bank Transfer, Mobile Wallet, Stripe*, PayPal*)
|
|
||||||
- Invoice generation and management
|
|
||||||
- Credit transaction tracking
|
|
||||||
- Multi-currency support (8 currencies)
|
|
||||||
- Manual payment approval workflow
|
|
||||||
|
|
||||||
**Enterprise Features:**
|
### Tech Stack
|
||||||
- Team collaboration and user management
|
- Django 5.x + DRF
|
||||||
- Activity logging and audit trails
|
|
||||||
- API access with rate limiting
|
|
||||||
- Comprehensive analytics and reporting
|
|
||||||
- Data export capabilities
|
|
||||||
|
|
||||||
### 🏗️ Technical Stack
|
|
||||||
|
|
||||||
**Backend:**
|
|
||||||
- Django 5.x + Django REST Framework
|
|
||||||
- PostgreSQL 14+ (production)
|
|
||||||
- Redis 7+ (cache and sessions)
|
|
||||||
- Celery + Celery Beat (task queue)
|
|
||||||
- JWT authentication
|
|
||||||
- Docker containerization
|
|
||||||
|
|
||||||
**Frontend:**
|
|
||||||
- React 19 + TypeScript
|
- React 19 + TypeScript
|
||||||
- Vite 6 (build tool)
|
- PostgreSQL + Redis
|
||||||
- Zustand 5 (state management)
|
- Celery + Celery Beat
|
||||||
- TailwindCSS 4 (styling)
|
- Docker deployment
|
||||||
- React Router 7 (routing)
|
|
||||||
- Custom component library
|
|
||||||
|
|
||||||
**AI Integration:**
|
|
||||||
- OpenAI GPT-4, GPT-4 Turbo, GPT-3.5 Turbo
|
|
||||||
- OpenAI DALL-E 3, DALL-E 2
|
|
||||||
- Runware image generation
|
|
||||||
|
|
||||||
### 📊 Platform Statistics
|
|
||||||
|
|
||||||
- 50+ backend API endpoints
|
|
||||||
- 100+ React components
|
|
||||||
- 15+ database models
|
|
||||||
- 20+ Celery tasks
|
|
||||||
- Support for 8 currencies
|
|
||||||
- 50+ industry templates
|
|
||||||
|
|
||||||
### 🔒 Security & Compliance
|
|
||||||
|
|
||||||
- Encrypted data at rest and in transit
|
|
||||||
- GDPR compliance ready
|
|
||||||
- Session integrity validation
|
|
||||||
- CORS and CSRF protection
|
|
||||||
- SQL injection prevention
|
|
||||||
- Rate limiting per scope
|
|
||||||
|
|
||||||
### 📚 Documentation
|
|
||||||
|
|
||||||
- Complete API documentation (OpenAPI 3.0)
|
|
||||||
- Frontend component library docs
|
|
||||||
- Admin guides and workflows
|
|
||||||
- Multi-tenancy architecture docs
|
|
||||||
- Payment workflow guides
|
|
||||||
|
|
||||||
### ⚠️ Known Limitations
|
|
||||||
|
|
||||||
- Stripe integration disabled (pending production credentials)
|
|
||||||
- PayPal integration disabled (pending production credentials)
|
|
||||||
- Some analytics features in development
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Tenancy Change Log - December 9, 2025
|
## Critical Bug Fixes - December 9, 2025
|
||||||
|
|
||||||
## Summary
|
|
||||||
This document tracks all changes made to the multi-tenancy system during the current staging session and the last 2 commits (4d13a570 and 72d0b6b0).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔥 Critical Fixes - December 9, 2025
|
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
- User swapping/logout issue - Redis sessions, no-cache auth backend, session integrity checks
|
- **User Swapping Issue**: Redis sessions + NoCacheModelBackend + session integrity checks
|
||||||
- useNavigate/useLocation HMR errors - Single Suspense boundary for Routes
|
- **HMR Navigation Errors**: Single top-level Suspense boundary for Routes
|
||||||
|
- **Registration Token Issue**: JWT tokens now returned on signup
|
||||||
### Added
|
|
||||||
- Custom `NoCacheModelBackend` authentication backend to prevent user object caching
|
|
||||||
- Session integrity validation in middleware (stores/verifies account_id and user_id per request)
|
|
||||||
|
|
||||||
### Changed
|
|
||||||
- Session storage from database to Redis cache (`SESSION_ENGINE = 'django.contrib.sessions.backends.cache'`)
|
|
||||||
- React Router Suspense from per-route to single top-level boundary
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔧 Recent Session Changes (Uncommitted)
|
## Pre-1.0 Development
|
||||||
|
|
||||||
### 1. Authentication & Signup Flow
|
### November-December 2025
|
||||||
**Fixed: JWT Token Generation in Registration**
|
- Initial development
|
||||||
- **Issue**: Users were immediately logged out after signup because tokens weren't being returned
|
- Multi-tenancy architecture
|
||||||
- **Root Cause**: Two separate `register` endpoints existed - one in `AuthViewSet` (unused) and one in `RegisterView` (actual endpoint)
|
- AI integration (OpenAI, DALL-E, Runware)
|
||||||
- **Fix**: Updated `RegisterView` in `backend/igny8_core/auth/urls.py` to generate and return JWT tokens
|
- WordPress integration
|
||||||
```python
|
- Automation pipeline
|
||||||
# Added token generation to RegisterView
|
- Billing system
|
||||||
access_token = generate_access_token(user, account)
|
|
||||||
refresh_token = generate_refresh_token(user, account)
|
|
||||||
# Return tokens in response data
|
|
||||||
```
|
|
||||||
- **Files Changed**: `backend/igny8_core/auth/urls.py`
|
|
||||||
- **Impact**: Users now stay logged in after successful registration
|
|
||||||
|
|
||||||
**Enhanced: Frontend Token Extraction**
|
|
||||||
- **Issue**: Frontend couldn't parse tokens from backend response structure
|
|
||||||
- **Fix**: Added multiple fallback paths in `authStore.ts` to handle nested response structure
|
|
||||||
```typescript
|
|
||||||
// Handle both data.tokens.access and data.data.tokens.access
|
|
||||||
const newToken = tokens.access || responseData.access || data.data?.tokens?.access
|
|
||||||
```
|
|
||||||
- **Files Changed**: `frontend/src/store/authStore.ts`
|
|
||||||
|
|
||||||
### 2. Payment Confirmation Modal
|
|
||||||
**Fixed: Invoice Amount Display**
|
|
||||||
- **Issue**: Amount showing as "PKR 0.00" in payment confirmation modal
|
|
||||||
- **Root Cause**: Frontend expected `total` field but backend returned `total_amount`
|
|
||||||
- **Fix**: Updated invoice API to return both fields for compatibility
|
|
||||||
```python
|
|
||||||
'total': str(invoice.total), # Alias for compatibility
|
|
||||||
'total_amount': str(invoice.total),
|
|
||||||
```
|
|
||||||
- **Files Changed**:
|
|
||||||
- `backend/igny8_core/business/billing/views.py`
|
|
||||||
- `frontend/src/components/billing/PaymentConfirmationModal.tsx`
|
|
||||||
- `frontend/src/components/billing/PendingPaymentBanner.tsx`
|
|
||||||
|
|
||||||
### 3. Payment Approval Workflow
|
|
||||||
**Fixed: Manual Status Change Not Triggering Account Activation**
|
|
||||||
- **Issue**: When admin changed payment status to "succeeded" in Django admin, it didn't activate account or add credits
|
|
||||||
- **Root Cause**: `save_model()` only set `approved_by` but didn't run the full approval workflow
|
|
||||||
- **Fix**: Enhanced `save_model()` in `PaymentAdmin` to trigger complete workflow:
|
|
||||||
- Update invoice status to 'paid'
|
|
||||||
- Activate subscription status to 'active'
|
|
||||||
- Activate account status to 'active'
|
|
||||||
- Add credits based on plan
|
|
||||||
- Prevent duplicate credit transactions
|
|
||||||
- **Files Changed**: `backend/igny8_core/modules/billing/admin.py`
|
|
||||||
- **Impact**: Admins can now manually approve payments in Django admin with full automation
|
|
||||||
|
|
||||||
### 4. Site Creation Permissions
|
|
||||||
**Fixed: Site Creation Failing Due to Permission Issues**
|
|
||||||
- **Issue**: Users couldn't create sites and were getting logged out
|
|
||||||
- **Root Cause**:
|
|
||||||
1. `SiteViewSet.get_permissions()` wasn't properly returning instances
|
|
||||||
2. Domain field validation rejected empty strings
|
|
||||||
- **Fixes Applied**:
|
|
||||||
- Updated `get_permissions()` to return instantiated permission classes
|
|
||||||
```python
|
|
||||||
return [IsAuthenticatedAndActive(), HasTenantAccess(), IsEditorOrAbove()]
|
|
||||||
```
|
|
||||||
- Modified domain validation to accept empty/None values
|
|
||||||
```python
|
|
||||||
if not value or value.strip() == '':
|
|
||||||
return None
|
|
||||||
```
|
|
||||||
- **Files Changed**:
|
|
||||||
- `backend/igny8_core/auth/views.py`
|
|
||||||
- `backend/igny8_core/auth/serializers.py`
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📦 Commit: 4d13a570 - Payment Methods and Configurations
|
## Changelog Guidelines
|
||||||
|
|
||||||
### Payment Method Configuration
|
When adding entries:
|
||||||
**Added: Global Payment Method Configurations**
|
1. Use version format: `v{major}.{minor}.{patch}`
|
||||||
- Created migration `0009_add_missing_payment_methods.py` to add:
|
2. Categorize: Added, Changed, Fixed, Removed, Security
|
||||||
- Bank Transfer (Manual) - Enabled for US, CA, GB, AU, PK, IN, EU
|
3. Include date
|
||||||
- Mobile Wallet (Manual) - Enabled for PK, IN
|
4. Reference related documentation if applicable
|
||||||
- Stripe (Disabled) - Configured for future use
|
5. Note breaking changes prominently
|
||||||
- PayPal (Disabled) - Configured for future use
|
|
||||||
|
|
||||||
**Added: Database Constraints and Indexes**
|
**Update this file after every release or significant change.**
|
||||||
- Migration `0010_add_database_constraints.py`:
|
|
||||||
- Added indexes on frequently queried fields
|
|
||||||
- Improved query performance for payment and invoice lookups
|
|
||||||
- Added constraints for data integrity
|
|
||||||
|
|
||||||
**Added: Webhook Configuration**
|
|
||||||
- Migration `0013_add_webhook_config.py`:
|
|
||||||
- Added webhook fields to `PaymentMethodConfig`:
|
|
||||||
- `webhook_url`
|
|
||||||
- `webhook_secret`
|
|
||||||
- `webhook_events` (JSON field)
|
|
||||||
- Prepared for Stripe/PayPal webhook integration
|
|
||||||
|
|
||||||
### Currency Conversion System
|
|
||||||
**Added: Multi-Currency Support**
|
|
||||||
- Created `backend/igny8_core/business/billing/utils/currency.py`:
|
|
||||||
- Currency multipliers for 8 countries (PKR, INR, GBP, CAD, AUD, EUR)
|
|
||||||
- `convert_usd_to_local()` function
|
|
||||||
- `format_currency()` function
|
|
||||||
- `get_currency_for_country()` mapping
|
|
||||||
|
|
||||||
**Updated: Invoice Creation with Local Currency**
|
|
||||||
- Modified `InvoiceService.create_subscription_invoice()`:
|
|
||||||
- Converts USD plan prices to local currency
|
|
||||||
- Stores original USD price in metadata
|
|
||||||
- Stores exchange rate for reference
|
|
||||||
- Modified `InvoiceService.create_credit_package_invoice()`:
|
|
||||||
- Same currency conversion logic
|
|
||||||
|
|
||||||
### Frontend Payment Components
|
|
||||||
**Added: PaymentHistory Component**
|
|
||||||
- Location: `frontend/src/components/billing/PaymentHistory.tsx`
|
|
||||||
- Features:
|
|
||||||
- Display user's payment history
|
|
||||||
- Status indicators (pending, succeeded, failed)
|
|
||||||
- Amount and currency display
|
|
||||||
- Manual reference and notes
|
|
||||||
|
|
||||||
**Enhanced: SignUpFormUnified**
|
|
||||||
- Updated plan display with currency conversion
|
|
||||||
- Dynamic payment method selection based on country
|
|
||||||
- Billing information collection for paid plans
|
|
||||||
- Payment confirmation modal integration
|
|
||||||
|
|
||||||
**Enhanced: PaymentConfirmationModal**
|
|
||||||
- Fixed amount display with proper currency
|
|
||||||
- Support for file upload (proof of payment)
|
|
||||||
- Transaction reference input
|
|
||||||
- Admin notes field
|
|
||||||
|
|
||||||
### Payment Workflow Services
|
|
||||||
**Added: Email Notification Service**
|
|
||||||
- Location: `backend/igny8_core/business/billing/services/email_service.py`
|
|
||||||
- Features:
|
|
||||||
- Payment confirmation emails
|
|
||||||
- Invoice emails
|
|
||||||
- Payment approval/rejection notifications
|
|
||||||
|
|
||||||
**Added: PDF Invoice Generation**
|
|
||||||
- Location: `backend/igny8_core/business/billing/services/pdf_service.py`
|
|
||||||
- Features:
|
|
||||||
- Generate PDF invoices
|
|
||||||
- Include company branding
|
|
||||||
- Line items and totals
|
|
||||||
- Payment instructions
|
|
||||||
|
|
||||||
**Added: Automated Tasks**
|
|
||||||
- `subscription_renewal.py`: Automatic subscription renewal
|
|
||||||
- `payment_retry.py`: Retry failed payments
|
|
||||||
|
|
||||||
### Testing
|
|
||||||
**Added: Comprehensive Test Suite**
|
|
||||||
- `test_payment_workflow.py`: End-to-end payment testing
|
|
||||||
- `test_payment_method_filtering.py`: Payment method availability tests
|
|
||||||
- `test_concurrency.py`: Concurrent payment handling tests
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📦 Commit: 72d0b6b0 - Tenancy Fixes
|
|
||||||
|
|
||||||
### Subscription Model Improvements
|
|
||||||
**Added: Database Constraints**
|
|
||||||
- Migration `0012_fix_subscription_constraints.py`:
|
|
||||||
- Ensured data integrity for subscription relationships
|
|
||||||
- Added proper foreign key constraints
|
|
||||||
|
|
||||||
**Simplified: Payment Status Flow**
|
|
||||||
- Migration `0007_simplify_payment_statuses.py`:
|
|
||||||
- Reduced payment statuses to core states
|
|
||||||
- Improved status transition logic
|
|
||||||
- Clearer admin workflow
|
|
||||||
|
|
||||||
### Model Enhancements
|
|
||||||
**Added: Invoice-Subscription Foreign Key**
|
|
||||||
- Migration `0008_add_invoice_subscription_fk.py`:
|
|
||||||
- Direct relationship between invoices and subscriptions
|
|
||||||
- Improved query performance
|
|
||||||
- Better data consistency
|
|
||||||
|
|
||||||
**Added: Payment-CreditTransaction Link**
|
|
||||||
- Migration `0012_add_payment_fk_to_credit_transaction.py`:
|
|
||||||
- Track which payment triggered credit addition
|
|
||||||
- Audit trail for credit transactions
|
|
||||||
- Prevent duplicate credit allocation
|
|
||||||
|
|
||||||
### Account Model Updates
|
|
||||||
**Enhanced: Billing Information Fields**
|
|
||||||
- Added comprehensive billing fields to Account model:
|
|
||||||
- `billing_email`
|
|
||||||
- `billing_address_line1`, `billing_address_line2`
|
|
||||||
- `billing_city`, `billing_state`, `billing_postal_code`
|
|
||||||
- `billing_country`
|
|
||||||
- `tax_id`
|
|
||||||
|
|
||||||
### Frontend Auth Improvements
|
|
||||||
**Enhanced: ProtectedRoute Component**
|
|
||||||
- Added 100ms initialization delay
|
|
||||||
- Improved token verification
|
|
||||||
- Better loading state management
|
|
||||||
- Prevents premature redirects
|
|
||||||
|
|
||||||
**Enhanced: SignUpFormSimplified**
|
|
||||||
- Streamlined UI for signup
|
|
||||||
- Better error handling
|
|
||||||
- Improved validation messages
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🗂️ Documentation Updates
|
|
||||||
|
|
||||||
### New Documentation
|
|
||||||
1. **PAYMENT-APPROVAL-FIXED.md**: Payment approval workflow guide
|
|
||||||
2. **ADMIN-PAYMENT-APPROVAL-GUIDE.md**: Step-by-step admin guide for approving payments
|
|
||||||
3. **SIGNUP-FIXES-DEC-9-2024.md**: Detailed signup flow fixes
|
|
||||||
|
|
||||||
### Updated Documentation Structure
|
|
||||||
```
|
|
||||||
multi-tenancy/
|
|
||||||
├── in-progress/
|
|
||||||
│ ├── ADMIN-PAYMENT-APPROVAL-GUIDE.md
|
|
||||||
│ ├── PAYMENT-WORKFLOW-QUICK-START.md
|
|
||||||
│ ├── SIGNUP-FIXES-DEC-9-2024.md
|
|
||||||
│ └── IMPLEMENTATION-STATUS.md
|
|
||||||
└── PAYMENT-APPROVAL-FIXED.md
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📊 Impact Summary
|
|
||||||
|
|
||||||
### Backend Changes
|
|
||||||
- **Models**: 6 new migrations, enhanced Account/Invoice/Payment/Subscription models
|
|
||||||
- **Services**: 3 new services (email, PDF, currency conversion)
|
|
||||||
- **Admin**: Enhanced payment approval workflow
|
|
||||||
- **API**: Fixed registration endpoint, improved invoice serialization
|
|
||||||
- **Tasks**: 2 new Celery tasks for automation
|
|
||||||
|
|
||||||
### Frontend Changes
|
|
||||||
- **Components**: 3 new/enhanced components (PaymentHistory, SignUpFormUnified, PaymentConfirmationModal)
|
|
||||||
- **Store**: Enhanced authStore with better token handling
|
|
||||||
- **Routing**: Improved ProtectedRoute with initialization delay
|
|
||||||
|
|
||||||
### Database Schema
|
|
||||||
- **New Fields**: 15+ new fields across models
|
|
||||||
- **New Indexes**: 8+ indexes for performance
|
|
||||||
- **New Constraints**: 5+ constraints for data integrity
|
|
||||||
- **New Foreign Keys**: 2 new relationships
|
|
||||||
|
|
||||||
### Testing
|
|
||||||
- **New Tests**: 3 comprehensive test files
|
|
||||||
- **Coverage**: Payment workflow, concurrency, method filtering
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🔍 Key Improvements
|
|
||||||
|
|
||||||
1. **Authentication Flow**: Seamless signup-to-login experience with proper JWT token handling
|
|
||||||
2. **Payment Processing**: Complete manual payment workflow with admin approval
|
|
||||||
3. **Multi-Currency**: Support for 8 currencies with automatic conversion
|
|
||||||
4. **Data Integrity**: Comprehensive constraints and foreign keys
|
|
||||||
5. **User Experience**: Better error handling, loading states, and feedback
|
|
||||||
6. **Admin Workflow**: One-click payment approval with automatic account activation
|
|
||||||
7. **Performance**: Added indexes on frequently queried fields
|
|
||||||
8. **Audit Trail**: Metadata tracking for all payment and credit transactions
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 🚀 Next Steps
|
|
||||||
|
|
||||||
### Immediate Priorities
|
|
||||||
1. Test complete signup → payment → activation flow
|
|
||||||
2. Verify currency conversion accuracy
|
|
||||||
3. Test site creation workflow
|
|
||||||
4. Validate webhook configurations
|
|
||||||
|
|
||||||
### Future Enhancements
|
|
||||||
1. Enable Stripe integration
|
|
||||||
2. Enable PayPal integration
|
|
||||||
3. Add automated payment retry logic
|
|
||||||
4. Implement subscription auto-renewal
|
|
||||||
5. Add invoice PDF email attachments
|
|
||||||
6. Create payment analytics dashboard
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 📝 Notes
|
|
||||||
|
|
||||||
### Breaking Changes
|
|
||||||
- None - all changes are backward compatible
|
|
||||||
|
|
||||||
### Deprecations
|
|
||||||
- Duplicate `AuthViewSet.register()` method (unused, kept for reference)
|
|
||||||
|
|
||||||
### Known Issues
|
|
||||||
- Workflow guide "dismissed" setting 404 error (non-critical, doesn't affect core functionality)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Last Updated**: December 9, 2024
|
|
||||||
**Session Duration**: ~4 hours
|
|
||||||
**Files Modified**: 51 files
|
|
||||||
**Lines Added**: 5,496
|
|
||||||
**Lines Removed**: 181
|
|
||||||
|
|||||||
@@ -1,356 +0,0 @@
|
|||||||
# Data Segregation: System vs User Data
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
This document categorizes all models in the Django admin sidebar to identify:
|
|
||||||
- **SYSTEM DATA**: Configuration, templates, and settings that must be preserved (pre-configured, production-ready data)
|
|
||||||
- **USER DATA**: Account-specific, tenant-specific, or test data that can be cleaned up during testing phase
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Accounts & Tenancy
|
|
||||||
|
|
||||||
| Model | Type | Description | Clean/Keep |
|
|
||||||
|-------|------|-------------|------------|
|
|
||||||
| Account | USER DATA | Customer accounts (test accounts during development) | ✅ CLEAN - Remove test accounts |
|
|
||||||
| User | USER DATA | User profiles linked to accounts | ✅ CLEAN - Remove test users |
|
|
||||||
| Site | USER DATA | Sites/domains owned by accounts | ✅ CLEAN - Remove test sites |
|
|
||||||
| Sector | USER DATA | Sectors within sites (account-specific) | ✅ CLEAN - Remove test sectors |
|
|
||||||
| SiteUserAccess | USER DATA | User permissions per site | ✅ CLEAN - Remove test access records |
|
|
||||||
|
|
||||||
**Summary**: All models are USER DATA - Safe to clean for fresh production start
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. Global Resources
|
|
||||||
|
|
||||||
| Model | Type | Description | Clean/Keep |
|
|
||||||
|-------|------|-------------|------------|
|
|
||||||
| Industry | SYSTEM DATA | Global industry taxonomy (e.g., Healthcare, Finance, Technology) | ⚠️ KEEP - Pre-configured industries |
|
|
||||||
| IndustrySector | SYSTEM DATA | Sub-categories within industries (e.g., Cardiology, Investment Banking) | ⚠️ KEEP - Pre-configured sectors |
|
|
||||||
| SeedKeyword | MIXED DATA | Seed keywords for industries - can be seeded or user-generated | ⚠️ REVIEW - Keep system seeds, remove test seeds |
|
|
||||||
|
|
||||||
**Summary**:
|
|
||||||
- **KEEP**: Industry and IndustrySector (global taxonomy)
|
|
||||||
- **REVIEW**: SeedKeyword - separate system defaults from test data
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. Plans and Billing
|
|
||||||
|
|
||||||
| Model | Type | Description | Clean/Keep |
|
|
||||||
|-------|------|-------------|------------|
|
|
||||||
| Plan | SYSTEM DATA | Subscription plans (Free, Pro, Enterprise, etc.) | ⚠️ KEEP - Production pricing tiers |
|
|
||||||
| Subscription | USER DATA | Active subscriptions per account | ✅ CLEAN - Remove test subscriptions |
|
|
||||||
| Invoice | USER DATA | Generated invoices for accounts | ✅ CLEAN - Remove test invoices |
|
|
||||||
| Payment | USER DATA | Payment records | ✅ CLEAN - Remove test payments |
|
|
||||||
| CreditPackage | SYSTEM DATA | Available credit packages for purchase | ⚠️ KEEP - Production credit offerings |
|
|
||||||
| PaymentMethodConfig | SYSTEM DATA | Supported payment methods (Stripe, PayPal) | ⚠️ KEEP - Production payment configs |
|
|
||||||
| AccountPaymentMethod | USER DATA | Saved payment methods per account | ✅ CLEAN - Remove test payment methods |
|
|
||||||
|
|
||||||
**Summary**:
|
|
||||||
- **KEEP**: Plan, CreditPackage, PaymentMethodConfig (system pricing/config)
|
|
||||||
- **CLEAN**: Subscription, Invoice, Payment, AccountPaymentMethod (user transactions)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. Credits
|
|
||||||
|
|
||||||
| Model | Type | Description | Clean/Keep |
|
|
||||||
|-------|------|-------------|------------|
|
|
||||||
| CreditTransaction | USER DATA | Credit add/subtract transactions | ✅ CLEAN - Remove test transactions |
|
|
||||||
| CreditUsageLog | USER DATA | Log of credit usage per operation | ✅ CLEAN - Remove test usage logs |
|
|
||||||
| CreditCostConfig | SYSTEM DATA | Cost configuration per operation type | ⚠️ KEEP - Production cost structure |
|
|
||||||
| PlanLimitUsage | USER DATA | Usage tracking per account/plan limits | ✅ CLEAN - Remove test usage data |
|
|
||||||
|
|
||||||
**Summary**:
|
|
||||||
- **KEEP**: CreditCostConfig (system cost rules)
|
|
||||||
- **CLEAN**: All transaction and usage logs (user activity)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5. Content Planning
|
|
||||||
|
|
||||||
| Model | Type | Description | Clean/Keep |
|
|
||||||
|-------|------|-------------|------------|
|
|
||||||
| Keywords | USER DATA | Keywords researched per site/sector | ✅ CLEAN - Remove test keywords |
|
|
||||||
| Clusters | USER DATA | Content clusters created per site | ✅ CLEAN - Remove test clusters |
|
|
||||||
| ContentIdeas | USER DATA | Content ideas generated for accounts | ✅ CLEAN - Remove test ideas |
|
|
||||||
|
|
||||||
**Summary**: All models are USER DATA - Safe to clean completely
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 6. Content Generation
|
|
||||||
|
|
||||||
| Model | Type | Description | Clean/Keep |
|
|
||||||
|-------|------|-------------|------------|
|
|
||||||
| Tasks | USER DATA | Content writing tasks assigned to users | ✅ CLEAN - Remove test tasks |
|
|
||||||
| Content | USER DATA | Generated content/articles | ✅ CLEAN - Remove test content |
|
|
||||||
| Images | USER DATA | Generated or uploaded images | ✅ CLEAN - Remove test images |
|
|
||||||
|
|
||||||
**Summary**: All models are USER DATA - Safe to clean completely
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 7. Taxonomy & Organization
|
|
||||||
|
|
||||||
| Model | Type | Description | Clean/Keep |
|
|
||||||
|-------|------|-------------|------------|
|
|
||||||
| ContentTaxonomy | USER DATA | Custom taxonomies (categories/tags) per site | ✅ CLEAN - Remove test taxonomies |
|
|
||||||
| ContentTaxonomyRelation | USER DATA | Relationships between content and taxonomies | ✅ CLEAN - Remove test relations |
|
|
||||||
| ContentClusterMap | USER DATA | Mapping of content to clusters | ✅ CLEAN - Remove test mappings |
|
|
||||||
| ContentAttribute | USER DATA | Custom attributes for content | ✅ CLEAN - Remove test attributes |
|
|
||||||
|
|
||||||
**Summary**: All models are USER DATA - Safe to clean completely
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 8. Publishing & Integration
|
|
||||||
|
|
||||||
| Model | Type | Description | Clean/Keep |
|
|
||||||
|-------|------|-------------|------------|
|
|
||||||
| SiteIntegration | USER DATA | WordPress/platform integrations per site | ✅ CLEAN - Remove test integrations |
|
|
||||||
| SyncEvent | USER DATA | Sync events between IGNY8 and external platforms | ✅ CLEAN - Remove test sync logs |
|
|
||||||
| PublishingRecord | USER DATA | Records of published content | ✅ CLEAN - Remove test publish records |
|
|
||||||
| PublishingChannel | SYSTEM DATA | Available publishing channels (WordPress, Ghost, etc.) | ⚠️ KEEP - Production channel configs |
|
|
||||||
| DeploymentRecord | USER DATA | Deployment history per account | ✅ CLEAN - Remove test deployments |
|
|
||||||
|
|
||||||
**Summary**:
|
|
||||||
- **KEEP**: PublishingChannel (system-wide channel definitions)
|
|
||||||
- **CLEAN**: All user-specific integration and sync data
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 9. AI & Automation
|
|
||||||
|
|
||||||
| Model | Type | Description | Clean/Keep |
|
|
||||||
|-------|------|-------------|------------|
|
|
||||||
| IntegrationSettings | MIXED DATA | API keys/settings for OpenAI, etc. | ⚠️ REVIEW - Keep system defaults, remove test configs |
|
|
||||||
| AIPrompt | SYSTEM DATA | AI prompt templates for content generation | ⚠️ KEEP - Production prompt library |
|
|
||||||
| Strategy | SYSTEM DATA | Content strategy templates | ⚠️ KEEP - Production strategy templates |
|
|
||||||
| AuthorProfile | SYSTEM DATA | Author persona templates | ⚠️ KEEP - Production author profiles |
|
|
||||||
| APIKey | USER DATA | User-generated API keys for platform access | ✅ CLEAN - Remove test API keys |
|
|
||||||
| WebhookConfig | USER DATA | Webhook configurations per account | ✅ CLEAN - Remove test webhooks |
|
|
||||||
| AutomationConfig | USER DATA | Automation rules per account/site | ✅ CLEAN - Remove test automations |
|
|
||||||
| AutomationRun | USER DATA | Execution history of automations | ✅ CLEAN - Remove test run logs |
|
|
||||||
|
|
||||||
**Summary**:
|
|
||||||
- **KEEP**: AIPrompt, Strategy, AuthorProfile (system templates)
|
|
||||||
- **REVIEW**: IntegrationSettings (separate system vs user API keys)
|
|
||||||
- **CLEAN**: APIKey, WebhookConfig, AutomationConfig, AutomationRun (user configs)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 10. System Settings
|
|
||||||
|
|
||||||
| Model | Type | Description | Clean/Keep |
|
|
||||||
|-------|------|-------------|------------|
|
|
||||||
| ContentType | SYSTEM DATA | Django ContentTypes (auto-managed) | ⚠️ KEEP - Django core system table |
|
|
||||||
| ContentTemplate | SYSTEM DATA | Content templates for generation | ⚠️ KEEP - Production templates |
|
|
||||||
| TaxonomyConfig | SYSTEM DATA | Taxonomy configuration rules | ⚠️ KEEP - Production taxonomy rules |
|
|
||||||
| SystemSetting | SYSTEM DATA | Global system settings | ⚠️ KEEP - Production system config |
|
|
||||||
| ContentTypeConfig | SYSTEM DATA | Content type definitions (blog post, landing page, etc.) | ⚠️ KEEP - Production content types |
|
|
||||||
| NotificationConfig | SYSTEM DATA | Notification templates and rules | ⚠️ KEEP - Production notification configs |
|
|
||||||
|
|
||||||
**Summary**: All models are SYSTEM DATA - Must be kept and properly seeded for production
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 11. Django Admin
|
|
||||||
|
|
||||||
| Model | Type | Description | Clean/Keep |
|
|
||||||
|-------|------|-------------|------------|
|
|
||||||
| Group | SYSTEM DATA | Permission groups (Admin, Editor, Viewer, etc.) | ⚠️ KEEP - Production role definitions |
|
|
||||||
| Permission | SYSTEM DATA | Django permissions (auto-managed) | ⚠️ KEEP - Django core system table |
|
|
||||||
| PasswordResetToken | USER DATA | Password reset tokens (temporary) | ✅ CLEAN - Remove expired tokens |
|
|
||||||
| Session | USER DATA | User session data | ✅ CLEAN - Remove old sessions |
|
|
||||||
|
|
||||||
**Summary**:
|
|
||||||
- **KEEP**: Group, Permission (system access control)
|
|
||||||
- **CLEAN**: PasswordResetToken, Session (temporary user data)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 12. Tasks & Logging
|
|
||||||
|
|
||||||
| Model | Type | Description | Clean/Keep |
|
|
||||||
|-------|------|-------------|------------|
|
|
||||||
| AITaskLog | USER DATA | Logs of AI operations per account | ✅ CLEAN - Remove test logs |
|
|
||||||
| AuditLog | USER DATA | Audit trail of user actions | ✅ CLEAN - Remove test audit logs |
|
|
||||||
| LogEntry | USER DATA | Django admin action logs | ✅ CLEAN - Remove test admin logs |
|
|
||||||
| TaskResult | USER DATA | Celery task execution results | ✅ CLEAN - Remove test task results |
|
|
||||||
| GroupResult | USER DATA | Celery group task results | ✅ CLEAN - Remove test group results |
|
|
||||||
|
|
||||||
**Summary**: All models are USER DATA - Safe to clean completely (logs/audit trails)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Summary Table: Data Segregation by Category
|
|
||||||
|
|
||||||
| Category | System Data Models | User Data Models | Mixed/Review |
|
|
||||||
|----------|-------------------|------------------|--------------|
|
|
||||||
| **Accounts & Tenancy** | 0 | 5 | 0 |
|
|
||||||
| **Global Resources** | 2 | 0 | 1 |
|
|
||||||
| **Plans and Billing** | 3 | 4 | 0 |
|
|
||||||
| **Credits** | 1 | 3 | 0 |
|
|
||||||
| **Content Planning** | 0 | 3 | 0 |
|
|
||||||
| **Content Generation** | 0 | 3 | 0 |
|
|
||||||
| **Taxonomy & Organization** | 0 | 4 | 0 |
|
|
||||||
| **Publishing & Integration** | 1 | 4 | 0 |
|
|
||||||
| **AI & Automation** | 3 | 4 | 1 |
|
|
||||||
| **System Settings** | 6 | 0 | 0 |
|
|
||||||
| **Django Admin** | 2 | 2 | 0 |
|
|
||||||
| **Tasks & Logging** | 0 | 5 | 0 |
|
|
||||||
| **TOTAL** | **18** | **37** | **2** |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Action Plan: Production Data Preparation
|
|
||||||
|
|
||||||
### Phase 1: Preserve System Data ⚠️
|
|
||||||
**Models to Keep & Seed Properly:**
|
|
||||||
|
|
||||||
1. **Global Taxonomy**
|
|
||||||
- Industry (pre-populate 10-15 major industries)
|
|
||||||
- IndustrySector (pre-populate 100+ sub-sectors)
|
|
||||||
- SeedKeyword (system-level seed keywords per industry)
|
|
||||||
|
|
||||||
2. **Pricing & Plans**
|
|
||||||
- Plan (Free, Starter, Pro, Enterprise tiers)
|
|
||||||
- CreditPackage (credit bundles for purchase)
|
|
||||||
- PaymentMethodConfig (Stripe, PayPal configs)
|
|
||||||
- CreditCostConfig (cost per operation type)
|
|
||||||
|
|
||||||
3. **Publishing Channels**
|
|
||||||
- PublishingChannel (WordPress, Ghost, Medium, etc.)
|
|
||||||
|
|
||||||
4. **AI & Content Templates**
|
|
||||||
- AIPrompt (100+ production-ready prompts)
|
|
||||||
- Strategy (content strategy templates)
|
|
||||||
- AuthorProfile (author persona library)
|
|
||||||
- ContentTemplate (article templates)
|
|
||||||
- ContentTypeConfig (blog post, landing page, etc.)
|
|
||||||
|
|
||||||
5. **System Configuration**
|
|
||||||
- SystemSetting (global platform settings)
|
|
||||||
- TaxonomyConfig (taxonomy rules)
|
|
||||||
- NotificationConfig (email/webhook templates)
|
|
||||||
|
|
||||||
6. **Access Control**
|
|
||||||
- Group (Admin, Editor, Viewer, Owner roles)
|
|
||||||
- Permission (Django-managed)
|
|
||||||
- ContentType (Django-managed)
|
|
||||||
|
|
||||||
### Phase 2: Clean User/Test Data ✅
|
|
||||||
**Models to Truncate/Delete:**
|
|
||||||
|
|
||||||
1. **Account Data**: Account, User, Site, Sector, SiteUserAccess
|
|
||||||
2. **Billing Transactions**: Subscription, Invoice, Payment, AccountPaymentMethod, CreditTransaction
|
|
||||||
3. **Content Data**: Keywords, Clusters, ContentIdeas, Tasks, Content, Images
|
|
||||||
4. **Taxonomy Relations**: ContentTaxonomy, ContentTaxonomyRelation, ContentClusterMap, ContentAttribute
|
|
||||||
5. **Integration Data**: SiteIntegration, SyncEvent, PublishingRecord, DeploymentRecord
|
|
||||||
6. **User Configs**: APIKey, WebhookConfig, AutomationConfig, AutomationRun
|
|
||||||
7. **Logs**: AITaskLog, AuditLog, LogEntry, TaskResult, GroupResult, CreditUsageLog, PlanLimitUsage, PasswordResetToken, Session
|
|
||||||
|
|
||||||
### Phase 3: Review Mixed Data ⚠️
|
|
||||||
**Models Requiring Manual Review:**
|
|
||||||
|
|
||||||
1. **SeedKeyword**: Separate system seeds from test data
|
|
||||||
2. **IntegrationSettings**: Keep system-level API configs, remove test account keys
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Database Cleanup Commands (Use with Caution)
|
|
||||||
|
|
||||||
### Safe Cleanup (Logs & Sessions)
|
|
||||||
```python
|
|
||||||
# Remove old logs (>90 days)
|
|
||||||
AITaskLog.objects.filter(created_at__lt=timezone.now() - timedelta(days=90)).delete()
|
|
||||||
CreditUsageLog.objects.filter(created_at__lt=timezone.now() - timedelta(days=90)).delete()
|
|
||||||
LogEntry.objects.filter(action_time__lt=timezone.now() - timedelta(days=90)).delete()
|
|
||||||
|
|
||||||
# Remove old sessions and tokens
|
|
||||||
Session.objects.filter(expire_date__lt=timezone.now()).delete()
|
|
||||||
PasswordResetToken.objects.filter(expires_at__lt=timezone.now()).delete()
|
|
||||||
|
|
||||||
# Remove old task results
|
|
||||||
TaskResult.objects.filter(date_done__lt=timezone.now() - timedelta(days=30)).delete()
|
|
||||||
```
|
|
||||||
|
|
||||||
### Full Test Data Cleanup (Development/Staging Only)
|
|
||||||
```python
|
|
||||||
# WARNING: Only run in development/staging environments
|
|
||||||
# This will delete ALL user-generated data
|
|
||||||
|
|
||||||
# User data
|
|
||||||
Account.objects.all().delete() # Cascades to most user data
|
|
||||||
User.objects.filter(is_superuser=False).delete()
|
|
||||||
|
|
||||||
# Remaining user data
|
|
||||||
SiteIntegration.objects.all().delete()
|
|
||||||
AutomationConfig.objects.all().delete()
|
|
||||||
APIKey.objects.all().delete()
|
|
||||||
WebhookConfig.objects.all().delete()
|
|
||||||
|
|
||||||
# Logs and history
|
|
||||||
AITaskLog.objects.all().delete()
|
|
||||||
AuditLog.objects.all().delete()
|
|
||||||
LogEntry.objects.all().delete()
|
|
||||||
TaskResult.objects.all().delete()
|
|
||||||
GroupResult.objects.all().delete()
|
|
||||||
```
|
|
||||||
|
|
||||||
### Verify System Data Exists
|
|
||||||
```python
|
|
||||||
# Check system data is properly seeded
|
|
||||||
print(f"Industries: {Industry.objects.count()}")
|
|
||||||
print(f"Plans: {Plan.objects.count()}")
|
|
||||||
print(f"AI Prompts: {AIPrompt.objects.count()}")
|
|
||||||
print(f"Strategies: {Strategy.objects.count()}")
|
|
||||||
print(f"Content Templates: {ContentTemplate.objects.count()}")
|
|
||||||
print(f"Publishing Channels: {PublishingChannel.objects.count()}")
|
|
||||||
print(f"Groups: {Group.objects.count()}")
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Recommendations
|
|
||||||
|
|
||||||
### Before Production Launch:
|
|
||||||
|
|
||||||
1. **Export System Data**: Export all SYSTEM DATA models to fixtures for reproducibility
|
|
||||||
```bash
|
|
||||||
python manage.py dumpdata igny8_core_auth.Industry > fixtures/industries.json
|
|
||||||
python manage.py dumpdata igny8_core_auth.Plan > fixtures/plans.json
|
|
||||||
python manage.py dumpdata system.AIPrompt > fixtures/prompts.json
|
|
||||||
# ... repeat for all system models
|
|
||||||
```
|
|
||||||
|
|
||||||
2. **Create Seed Script**: Create management command to populate fresh database with system data
|
|
||||||
```bash
|
|
||||||
python manage.py seed_system_data
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Database Snapshot**: Take snapshot after system data is seeded, before any user data
|
|
||||||
|
|
||||||
4. **Separate Databases**: Consider separate staging database with full test data vs production with clean start
|
|
||||||
|
|
||||||
5. **Data Migration Plan**:
|
|
||||||
- If migrating from old system: Only migrate Account, User, Content, and critical user data
|
|
||||||
- Leave test data behind in old system
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Next Steps
|
|
||||||
|
|
||||||
1. ✅ Review this document and confirm data segregation logic
|
|
||||||
2. ⚠️ Create fixtures/seeds for all 18 SYSTEM DATA models
|
|
||||||
3. ⚠️ Review 2 MIXED DATA models (SeedKeyword, IntegrationSettings)
|
|
||||||
4. ✅ Create cleanup script for 37 USER DATA models
|
|
||||||
5. ✅ Test cleanup script in staging environment
|
|
||||||
6. ✅ Execute cleanup before production launch
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
*Generated: December 20, 2025*
|
|
||||||
*Purpose: Production data preparation and test data cleanup*
|
|
||||||
223
IGNY8-APP.md
Normal file
223
IGNY8-APP.md
Normal file
@@ -0,0 +1,223 @@
|
|||||||
|
# IGNY8 - AI-Powered SEO Content Platform
|
||||||
|
|
||||||
|
**Version:** 1.0.5
|
||||||
|
**Last Updated:** December 25, 2025
|
||||||
|
**Status:** Production Ready
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What is IGNY8?
|
||||||
|
|
||||||
|
IGNY8 is an enterprise-grade AI-powered content platform that helps businesses create, manage, and publish SEO-optimized content at scale. It combines artificial intelligence with workflow automation to transform keyword research into published articles with minimal manual effort.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Platform Overview
|
||||||
|
|
||||||
|
| Aspect | Details |
|
||||||
|
|--------|---------|
|
||||||
|
| Type | Full-stack SaaS Platform |
|
||||||
|
| Architecture | Multi-tenant cloud application |
|
||||||
|
| Target Users | Content marketers, SEO agencies, digital publishers |
|
||||||
|
| Deployment | Docker-based, cloud-hosted |
|
||||||
|
| Pricing | Credit-based usage with subscription plans |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Core Capabilities
|
||||||
|
|
||||||
|
### 1. AI Content Generation
|
||||||
|
|
||||||
|
Transform keywords into fully-formed, SEO-optimized articles:
|
||||||
|
|
||||||
|
- **GPT-4 Powered Writing**: High-quality, contextually-aware content
|
||||||
|
- **Smart Structuring**: Proper headings, paragraphs, lists automatically
|
||||||
|
- **SEO Optimization**: Meta titles, descriptions, keyword density
|
||||||
|
- **Customizable Length**: 500 to 5,000+ words per article
|
||||||
|
- **Multiple Content Types**: Blog posts, guides, comparisons, reviews
|
||||||
|
|
||||||
|
### 2. Intelligent Keyword Management
|
||||||
|
|
||||||
|
Comprehensive keyword research and organization:
|
||||||
|
|
||||||
|
- **Bulk Import**: Upload thousands of keywords via CSV
|
||||||
|
- **AI Clustering**: Automatically group related keywords
|
||||||
|
- **Global Database**: 50+ industries with seed keywords
|
||||||
|
- **Intent Classification**: Informational, commercial, transactional
|
||||||
|
- **Metrics Tracking**: Volume, difficulty, CPC
|
||||||
|
|
||||||
|
### 3. Content Idea Generation
|
||||||
|
|
||||||
|
Transform clusters into actionable content briefs:
|
||||||
|
|
||||||
|
- **AI-Powered**: Generate ideas from keyword clusters
|
||||||
|
- **Structured Briefs**: Title, keywords, angle, outline
|
||||||
|
- **Word Count Estimates**: Based on competition analysis
|
||||||
|
- **Priority Scoring**: Rank by SEO potential
|
||||||
|
|
||||||
|
### 4. Image Generation
|
||||||
|
|
||||||
|
Dual AI providers for visual content:
|
||||||
|
|
||||||
|
- **DALL-E 3**: High-quality image generation
|
||||||
|
- **Runware**: Alternative provider for variety
|
||||||
|
- **Smart Prompts**: Extracted from content context
|
||||||
|
- **Multiple Types**: Featured, in-article, mobile/desktop
|
||||||
|
|
||||||
|
### 5. Automation Pipeline
|
||||||
|
|
||||||
|
Complete 7-stage automated workflow:
|
||||||
|
|
||||||
|
```
|
||||||
|
Keywords → Clusters → Ideas → Tasks → Content → Image Prompts → Images
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Scheduled Runs**: Daily, weekly, or monthly
|
||||||
|
- **Configurable**: Batch sizes, delays, stages
|
||||||
|
- **Credit Estimation**: Know costs before running
|
||||||
|
- **Pause/Resume**: Control running automation
|
||||||
|
|
||||||
|
### 6. WordPress Integration
|
||||||
|
|
||||||
|
Seamless publishing to WordPress:
|
||||||
|
|
||||||
|
- **One-Click Publish**: Push content directly
|
||||||
|
- **Two-Way Sync**: Import and export content
|
||||||
|
- **Taxonomy Sync**: Categories and tags
|
||||||
|
- **Image Upload**: Featured and in-article
|
||||||
|
- **Multiple Sites**: Manage many WordPress sites
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Module Status
|
||||||
|
|
||||||
|
| Module | Status | Description |
|
||||||
|
|--------|--------|-------------|
|
||||||
|
| **Planner** | ✅ Active | Keyword management, clustering, ideas |
|
||||||
|
| **Writer** | ✅ Active | Task management, content generation, images |
|
||||||
|
| **Automation** | ✅ Active | 7-stage automated pipeline |
|
||||||
|
| **Billing** | ✅ Active | Credits, plans, payments |
|
||||||
|
| **Integrations** | ✅ Active | WordPress sync |
|
||||||
|
| **Settings** | ✅ Active | AI config, prompts, modules |
|
||||||
|
| **Publisher** | ✅ Active | Publishing pipeline |
|
||||||
|
| **Linker** | ⏸️ Inactive | Internal linking (available but disabled) |
|
||||||
|
| **Optimizer** | ⏸️ Inactive | Content optimization (available but disabled) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Plans & Pricing
|
||||||
|
|
||||||
|
### Subscription Plans
|
||||||
|
|
||||||
|
| Plan | Sites | Users | Credits/Month | Best For |
|
||||||
|
|------|-------|-------|---------------|----------|
|
||||||
|
| Free | 1 | 1 | 100 | Trial |
|
||||||
|
| Starter | 3 | 3 | 1,000 | Individuals |
|
||||||
|
| Growth | 10 | 10 | 5,000 | Small teams |
|
||||||
|
| Scale | Unlimited | Unlimited | 25,000 | Agencies |
|
||||||
|
|
||||||
|
### Credit Costs (Typical)
|
||||||
|
|
||||||
|
| Operation | Credits |
|
||||||
|
|-----------|---------|
|
||||||
|
| Keyword Clustering (batch) | 10 |
|
||||||
|
| Content Idea | 2 |
|
||||||
|
| Content (per 100 words) | 5 |
|
||||||
|
| Image (basic) | 3 |
|
||||||
|
| Image (premium) | 5 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## User Roles
|
||||||
|
|
||||||
|
| Role | Access Level |
|
||||||
|
|------|--------------|
|
||||||
|
| Admin | Full account access |
|
||||||
|
| Manager | Content + billing view |
|
||||||
|
| Editor | AI content, clusters, tasks |
|
||||||
|
| Viewer | Read-only dashboards |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Key Workflows
|
||||||
|
|
||||||
|
### Manual Content Creation
|
||||||
|
|
||||||
|
1. Import keywords or select from database
|
||||||
|
2. Run AI clustering to group keywords
|
||||||
|
3. Generate content ideas from clusters
|
||||||
|
4. Create tasks from ideas
|
||||||
|
5. Generate content with AI
|
||||||
|
6. Generate images for content
|
||||||
|
7. Review and edit
|
||||||
|
8. Publish to WordPress
|
||||||
|
|
||||||
|
### Automated Content Pipeline
|
||||||
|
|
||||||
|
1. Configure automation settings
|
||||||
|
2. Set schedule (daily/weekly/monthly)
|
||||||
|
3. System runs 7 stages automatically
|
||||||
|
4. Content queued for review
|
||||||
|
5. Approve and publish
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Security & Compliance
|
||||||
|
|
||||||
|
- Encrypted data at rest and in transit
|
||||||
|
- Multi-tenant data isolation
|
||||||
|
- Role-based access control
|
||||||
|
- Session management with Redis
|
||||||
|
- GDPR compliance ready
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Current Limitations
|
||||||
|
|
||||||
|
| Item | Status |
|
||||||
|
|------|--------|
|
||||||
|
| Stripe payments | Pending production credentials |
|
||||||
|
| PayPal payments | Pending production credentials |
|
||||||
|
| Linker module | Disabled by default |
|
||||||
|
| Optimizer module | Disabled by default |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Technical Details
|
||||||
|
|
||||||
|
For technical documentation, see: `/docs/INDEX.md`
|
||||||
|
|
||||||
|
| Component | Technology |
|
||||||
|
|-----------|------------|
|
||||||
|
| Backend | Django 5.x, Python |
|
||||||
|
| Frontend | React 19, TypeScript |
|
||||||
|
| Database | PostgreSQL |
|
||||||
|
| Cache | Redis |
|
||||||
|
| AI | OpenAI GPT-4, DALL-E 3 |
|
||||||
|
| Deployment | Docker |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Support & Resources
|
||||||
|
|
||||||
|
| Resource | Location |
|
||||||
|
|----------|----------|
|
||||||
|
| Technical Docs | `/docs/INDEX.md` |
|
||||||
|
| API Reference | `/docs/20-API/ENDPOINTS.md` |
|
||||||
|
| Changelog | `/CHANGELOG.md` |
|
||||||
|
| Rules | `/RULES.md` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Version History
|
||||||
|
|
||||||
|
- **v1.0.5** (Dec 12, 2025): Purchase Credits tab, UI reorganization
|
||||||
|
- **v1.0.4** (Dec 12, 2025): Credit Activity tab, Cost Reference panel
|
||||||
|
- **v1.0.3** (Dec 12, 2025): Usage Limits colors, Cost Breakdown
|
||||||
|
- **v1.0.2** (Dec 12, 2025): Design system enforcement
|
||||||
|
- **v1.0.1** (Dec 12, 2025): Plan limits UI
|
||||||
|
- **v1.0.0** (Dec 12, 2025): Initial production release
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*This document provides a non-technical overview. For technical implementation details, refer to the docs folder.*
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,223 +0,0 @@
|
|||||||
# Integration Settings Workflow & Data Flow
|
|
||||||
|
|
||||||
## Part 1: How Global Settings Load on Frontend
|
|
||||||
|
|
||||||
### Admin Configures Global Settings
|
|
||||||
**URL**: `https://api.igny8.com/admin/system/globalintegrationsettings/1/change/`
|
|
||||||
|
|
||||||
**What's Stored**:
|
|
||||||
- Platform-wide API keys (OpenAI, DALL-E, Runware)
|
|
||||||
- Default model selections (gpt-4o-mini, dall-e-3, runware:97@1)
|
|
||||||
- Default parameters (temperature: 0.7, max_tokens: 8192)
|
|
||||||
- Default image settings (size, quality, style)
|
|
||||||
|
|
||||||
**Who Can Access**: Only platform administrators
|
|
||||||
|
|
||||||
### Normal User Opens Integration Page
|
|
||||||
**URL**: `https://app.igny8.com/settings/integration`
|
|
||||||
|
|
||||||
**What Happens**:
|
|
||||||
|
|
||||||
1. **Frontend Request**:
|
|
||||||
- User browser requests: `GET /api/v1/system/settings/integrations/openai/`
|
|
||||||
- User browser requests: `GET /api/v1/system/settings/integrations/image_generation/`
|
|
||||||
|
|
||||||
2. **Backend Processing** (`integration_views.py` - `get_settings()` method):
|
|
||||||
- Checks if user's account has custom overrides in `IntegrationSettings` table
|
|
||||||
- Gets global defaults from `GlobalIntegrationSettings` singleton
|
|
||||||
- Merges data with this priority:
|
|
||||||
- If account has overrides → use account settings
|
|
||||||
- If no overrides → use global defaults
|
|
||||||
- **NEVER returns API keys** (security)
|
|
||||||
|
|
||||||
3. **Response to Frontend**:
|
|
||||||
```
|
|
||||||
{
|
|
||||||
"id": "openai",
|
|
||||||
"enabled": true,
|
|
||||||
"model": "gpt-4o-mini", // From global OR account override
|
|
||||||
"temperature": 0.7, // From global OR account override
|
|
||||||
"max_tokens": 8192, // From global OR account override
|
|
||||||
"using_global": true // Flag: true if using defaults
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
4. **Frontend Display**:
|
|
||||||
- Shows current model selection
|
|
||||||
- Shows "Using platform defaults" badge if `using_global: true`
|
|
||||||
- Shows "Custom settings" badge if `using_global: false`
|
|
||||||
- User can change model, temperature, etc.
|
|
||||||
- **API key status is NOT shown** (user cannot see/change platform keys)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Part 2: How User Changes Are Saved
|
|
||||||
|
|
||||||
### User Changes Settings on Frontend
|
|
||||||
|
|
||||||
1. **User Actions**:
|
|
||||||
- Opens settings modal
|
|
||||||
- Changes model from `gpt-4o-mini` to `gpt-4o`
|
|
||||||
- Changes temperature from `0.7` to `0.8`
|
|
||||||
- Clicks "Save"
|
|
||||||
|
|
||||||
2. **Frontend Request**:
|
|
||||||
- Sends: `PUT /api/v1/system/settings/integrations/openai/`
|
|
||||||
- Body: `{"model": "gpt-4o", "temperature": 0.8, "max_tokens": 8192}`
|
|
||||||
|
|
||||||
3. **Backend Processing** (`integration_views.py` - `save_settings()` method):
|
|
||||||
- **CRITICAL SECURITY**: Strips ANY API keys from request (apiKey, api_key, openai_api_key, etc.)
|
|
||||||
- Validates account exists
|
|
||||||
- Builds clean config with ONLY allowed overrides:
|
|
||||||
- For OpenAI: model, temperature, max_tokens
|
|
||||||
- For Image: service, model, image_quality, image_style, sizes
|
|
||||||
- Saves to `IntegrationSettings` table:
|
|
||||||
```
|
|
||||||
account_id: 123
|
|
||||||
integration_type: "openai"
|
|
||||||
config: {"model": "gpt-4o", "temperature": 0.8, "max_tokens": 8192}
|
|
||||||
is_active: true
|
|
||||||
```
|
|
||||||
|
|
||||||
4. **Database Structure**:
|
|
||||||
- **GlobalIntegrationSettings** (1 row, pk=1):
|
|
||||||
- Contains: API keys + default settings
|
|
||||||
- Used by: ALL accounts for API keys
|
|
||||||
|
|
||||||
- **IntegrationSettings** (multiple rows):
|
|
||||||
- Row per account per integration type
|
|
||||||
- Contains: ONLY overrides (no API keys)
|
|
||||||
- Example:
|
|
||||||
```
|
|
||||||
id | account_id | integration_type | config
|
|
||||||
100 | 123 | openai | {"model": "gpt-4o", "temperature": 0.8}
|
|
||||||
101 | 456 | openai | {"model": "gpt-4.1", "max_tokens": 4000}
|
|
||||||
102 | 123 | image_generation| {"service": "runware", "model": "runware:100@1"}
|
|
||||||
```
|
|
||||||
|
|
||||||
5. **Next Request from User**:
|
|
||||||
- Frontend requests: `GET /api/v1/system/settings/integrations/openai/`
|
|
||||||
- Backend finds IntegrationSettings row for account 123
|
|
||||||
- Returns: `{"model": "gpt-4o", "temperature": 0.8, "using_global": false}`
|
|
||||||
- User sees their custom settings
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Data Flow Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ ADMIN SIDE │
|
|
||||||
│ https://api.igny8.com/admin/ │
|
|
||||||
│ │
|
|
||||||
│ GlobalIntegrationSettings (pk=1) │
|
|
||||||
│ ├── openai_api_key: "sk-xxx" ← Platform-wide │
|
|
||||||
│ ├── openai_model: "gpt-4o-mini" ← Default │
|
|
||||||
│ ├── openai_temperature: 0.7 ← Default │
|
|
||||||
│ ├── dalle_api_key: "sk-xxx" ← Platform-wide │
|
|
||||||
│ ├── runware_api_key: "xxx" ← Platform-wide │
|
|
||||||
│ └── image_quality: "standard" ← Default │
|
|
||||||
└─────────────────────────────────────────────────────────────┘
|
|
||||||
│
|
|
||||||
│ Backend reads from
|
|
||||||
↓
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ BACKEND API LAYER │
|
|
||||||
│ integration_views.py │
|
|
||||||
│ │
|
|
||||||
│ get_settings(): │
|
|
||||||
│ 1. Load GlobalIntegrationSettings (for defaults) │
|
|
||||||
│ 2. Check IntegrationSettings (for account overrides) │
|
|
||||||
│ 3. Merge: account overrides > global defaults │
|
|
||||||
│ 4. Return to frontend (NO API keys) │
|
|
||||||
│ │
|
|
||||||
│ save_settings(): │
|
|
||||||
│ 1. Receive request from frontend │
|
|
||||||
│ 2. Strip ALL API keys (security) │
|
|
||||||
│ 3. Save ONLY overrides to IntegrationSettings │
|
|
||||||
└─────────────────────────────────────────────────────────────┘
|
|
||||||
│
|
|
||||||
│ API sends data
|
|
||||||
↓
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ FRONTEND - USER SIDE │
|
|
||||||
│ https://app.igny8.com/settings/integration │
|
|
||||||
│ │
|
|
||||||
│ User sees: │
|
|
||||||
│ ├── Model: gpt-4o-mini (dropdown) │
|
|
||||||
│ ├── Temperature: 0.7 (slider) │
|
|
||||||
│ ├── Status: ✓ Connected (test connection works) │
|
|
||||||
│ └── Badge: "Using platform defaults" │
|
|
||||||
│ │
|
|
||||||
│ User CANNOT see: │
|
|
||||||
│ ✗ API keys (security) │
|
|
||||||
│ ✗ Platform configuration │
|
|
||||||
└─────────────────────────────────────────────────────────────┘
|
|
||||||
│
|
|
||||||
│ User changes settings
|
|
||||||
↓
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ IntegrationSettings Table │
|
|
||||||
│ (Per-account overrides - NO API KEYS) │
|
|
||||||
│ │
|
|
||||||
│ Account 123: │
|
|
||||||
│ ├── openai: {"model": "gpt-4o", "temperature": 0.8} │
|
|
||||||
│ └── image_generation: {"service": "runware"} │
|
|
||||||
│ │
|
|
||||||
│ Account 456: │
|
|
||||||
│ ├── openai: {"model": "gpt-4.1"} │
|
|
||||||
│ └── image_generation: (no row = uses global defaults) │
|
|
||||||
└─────────────────────────────────────────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Important Security Rules
|
|
||||||
|
|
||||||
1. **API Keys Flow**:
|
|
||||||
- Admin sets → GlobalIntegrationSettings
|
|
||||||
- Backend uses → For ALL accounts
|
|
||||||
- Frontend NEVER sees → Security
|
|
||||||
- Users NEVER save → Stripped by backend
|
|
||||||
|
|
||||||
2. **Settings Flow**:
|
|
||||||
- Admin sets defaults → GlobalIntegrationSettings
|
|
||||||
- Users customize → IntegrationSettings (overrides only)
|
|
||||||
- Backend merges → Global defaults + account overrides
|
|
||||||
- Frontend displays → Merged result (no keys)
|
|
||||||
|
|
||||||
3. **Free Plan Restriction**:
|
|
||||||
- Cannot create IntegrationSettings rows
|
|
||||||
- Must use global defaults only
|
|
||||||
- Enforced at frontend (UI disabled)
|
|
||||||
- TODO: Add backend validation
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Example Scenarios
|
|
||||||
|
|
||||||
### Scenario 1: New User First Visit
|
|
||||||
- User has NO IntegrationSettings row
|
|
||||||
- Backend returns global defaults
|
|
||||||
- `using_global: true`
|
|
||||||
- User sees platform defaults
|
|
||||||
- API operations use platform API key
|
|
||||||
|
|
||||||
### Scenario 2: User Customizes Model
|
|
||||||
- User changes model to "gpt-4o"
|
|
||||||
- Frontend sends: `{"model": "gpt-4o"}`
|
|
||||||
- Backend creates IntegrationSettings row
|
|
||||||
- Next visit: `using_global: false`
|
|
||||||
- API operations use platform API key + user's model choice
|
|
||||||
|
|
||||||
### Scenario 3: User Resets to Default
|
|
||||||
- Frontend sends: `{"model": "gpt-4o-mini"}` (same as global)
|
|
||||||
- Backend still saves override row
|
|
||||||
- Alternative: Delete row to truly use global
|
|
||||||
- TODO: Add "Reset to defaults" button
|
|
||||||
|
|
||||||
### Scenario 4: Admin Changes Global Default
|
|
||||||
- Admin changes global model to "gpt-4.1"
|
|
||||||
- Users WITH overrides: See their custom model
|
|
||||||
- Users WITHOUT overrides: See new "gpt-4.1" default
|
|
||||||
- All users: Use platform API key
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
## 🔴 AI FUunctions progress modals texts and counts to be fixed
|
|
||||||
|
|
||||||
## 🔴 AUTOAMTION queue when run manualy completed count to be fixed, and progress abr to be imrpoved and fixed based on actual stage and all other data have bugs
|
|
||||||
|
|
||||||
## 🔴 Align prompts with teh strategy
|
|
||||||
|
|
||||||
## 🔴 user randomly logs out often
|
|
||||||
|
|
||||||
## 🔴 MArketing site cotnetn
|
|
||||||
|
|
||||||
## 🔴 docuementation adn help update
|
|
||||||
341
PRODUCTION-READINESS-PLAN.md
Normal file
341
PRODUCTION-READINESS-PLAN.md
Normal file
@@ -0,0 +1,341 @@
|
|||||||
|
# Production Readiness Plan
|
||||||
|
|
||||||
|
**Created:** December 25, 2025
|
||||||
|
**Last Updated:** December 25, 2025
|
||||||
|
**Goal:** Ship to production with simplified UI while keeping backend unchanged
|
||||||
|
**Status:** ✅ CORE CHANGES COMPLETE
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Implementation Status
|
||||||
|
|
||||||
|
### ✅ Completed Changes
|
||||||
|
|
||||||
|
| File | Change | Status |
|
||||||
|
|------|--------|--------|
|
||||||
|
| `layout/AppLayout.tsx` | Header shows "Content: X/Y" | ✅ Done |
|
||||||
|
| `pages/Dashboard/Home.tsx` | Credit widget → "Content This Month" | ✅ Done |
|
||||||
|
| `pages/Help/Help.tsx` | FAQ updated | ✅ Done |
|
||||||
|
| `pages/Automation/AutomationPage.tsx` | Error messages updated | ✅ Done |
|
||||||
|
| `components/auth/SignUpForm*.tsx` | 3 files - removed credit text | ✅ Done |
|
||||||
|
| `pages/Settings/Plans.tsx` | Shows "Pages/Articles per month" | ✅ Done |
|
||||||
|
| `pages/account/PlansAndBillingPage.tsx` | Tabs simplified, Purchase Credits hidden | ✅ Done |
|
||||||
|
| `pages/Payment.tsx` | Updated pricing display | ✅ Done |
|
||||||
|
| `pages/Optimizer/Dashboard.tsx` | Removed credits card | ✅ Done |
|
||||||
|
| `pages/Settings/CreditsAndBilling.tsx` | Renamed to "Usage & Billing" | ✅ Done |
|
||||||
|
| `pages/Billing/Credits.tsx` | Renamed to "Content Usage" | ✅ Done |
|
||||||
|
| `components/billing/BillingBalancePanel.tsx` | "Usage Overview" | ✅ Done |
|
||||||
|
| `components/dashboard/CreditBalanceWidget.tsx` | "Content Usage" | ✅ Done |
|
||||||
|
| `components/dashboard/UsageChartWidget.tsx` | "Content Created" | ✅ Done |
|
||||||
|
| `components/ui/pricing-table/index.tsx` | "Content pieces/month" | ✅ Done |
|
||||||
|
| `marketing/pages/Pricing.tsx` | "Usage Dashboard" | ✅ Done |
|
||||||
|
| `docs/40-WORKFLOWS/CREDIT-SYSTEM.md` | Updated docs | ✅ Done |
|
||||||
|
|
||||||
|
### Hidden from Regular Users
|
||||||
|
|
||||||
|
| Feature | Status |
|
||||||
|
|---------|--------|
|
||||||
|
| Purchase Credits page | Hidden (commented out) |
|
||||||
|
| Credit Cost Breakdown | Hidden |
|
||||||
|
| Credit Packages grid | Hidden |
|
||||||
|
| Detailed credit analytics | Hidden |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## The Strategy: Abstraction Layer
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────────┐
|
||||||
|
│ WHAT CHANGES │
|
||||||
|
├─────────────────────────────────────────────────────────────────┤
|
||||||
|
│ │
|
||||||
|
│ BACKEND (NO CHANGES) FRONTEND (CHANGES) │
|
||||||
|
│ ────────────────────── ────────────────── │
|
||||||
|
│ • Credit deduction ✓ • Hide credit numbers ✅ │
|
||||||
|
│ • Usage tracking ✓ • Show "Content Pieces" ✅ │
|
||||||
|
│ • Plan limits ✓ • Simplify terminology ✅ │
|
||||||
|
│ • API responses ✓ • Remove purchase credits ✅ │
|
||||||
|
│ │
|
||||||
|
└─────────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
**Key Insight:** Backend tracks `content_credits`. User sees this as "Content Pieces".
|
||||||
|
No conversion needed - just rename the display.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1: Core Abstraction (billingStore)
|
||||||
|
|
||||||
|
### File: `frontend/src/store/billingStore.ts`
|
||||||
|
|
||||||
|
Add computed properties that translate credits → content pieces:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Add to BillingState interface
|
||||||
|
interface BillingState {
|
||||||
|
// ... existing ...
|
||||||
|
|
||||||
|
// Computed for simple UI
|
||||||
|
getContentPiecesRemaining: () => number;
|
||||||
|
getContentPiecesUsed: () => number;
|
||||||
|
getContentPiecesTotal: () => number;
|
||||||
|
}
|
||||||
|
|
||||||
|
// In the store
|
||||||
|
getContentPiecesRemaining: () => {
|
||||||
|
const balance = get().balance;
|
||||||
|
// content_credits = content pieces (1:1 mapping)
|
||||||
|
return balance?.credits_remaining ?? 0;
|
||||||
|
},
|
||||||
|
|
||||||
|
getContentPiecesUsed: () => {
|
||||||
|
const balance = get().balance;
|
||||||
|
return balance?.credits_used_this_month ?? 0;
|
||||||
|
},
|
||||||
|
|
||||||
|
getContentPiecesTotal: () => {
|
||||||
|
const balance = get().balance;
|
||||||
|
return balance?.plan_credits_per_month ?? 0;
|
||||||
|
},
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 2: Header Metrics Update
|
||||||
|
|
||||||
|
### File: `frontend/src/layout/AppLayout.tsx`
|
||||||
|
|
||||||
|
**Current (line ~138):**
|
||||||
|
```typescript
|
||||||
|
{
|
||||||
|
label: 'Credits',
|
||||||
|
value: balance.credits.toLocaleString(),
|
||||||
|
// ...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Change to:**
|
||||||
|
```typescript
|
||||||
|
{
|
||||||
|
label: 'Content',
|
||||||
|
value: `${balance.credits_remaining}/${balance.plan_credits_per_month}`,
|
||||||
|
tooltip: 'Content pieces remaining this month'
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 3: Pages to Update
|
||||||
|
|
||||||
|
### Priority 1: Remove/Hide These Pages
|
||||||
|
|
||||||
|
| Page | File | Action |
|
||||||
|
|------|------|--------|
|
||||||
|
| Purchase Credits | `pages/account/PurchaseCreditsPage.tsx` | Hide from nav, keep for admin |
|
||||||
|
| Credits Tab | `pages/Billing/Credits.tsx` | Remove tab from billing |
|
||||||
|
| Credits & Billing | `pages/Settings/CreditsAndBilling.tsx` | Rename to "Usage" |
|
||||||
|
|
||||||
|
### Priority 2: Update Text/Labels
|
||||||
|
|
||||||
|
| Page | File | Changes |
|
||||||
|
|------|------|---------|
|
||||||
|
| Dashboard | `pages/Dashboard/Home.tsx` | "Credits" → "Content Pieces" |
|
||||||
|
| Plans & Billing | `pages/account/PlansAndBillingPage.tsx` | Remove Credits tab, simplify |
|
||||||
|
| Usage Analytics | `pages/account/UsageAnalyticsPage.tsx` | "Credits" → "Content" |
|
||||||
|
| Automation | `pages/Automation/AutomationPage.tsx` | "credits" → "content pieces" |
|
||||||
|
| Help | `pages/Help/Help.tsx` | Update FAQ text |
|
||||||
|
|
||||||
|
### Priority 3: Marketing Pages
|
||||||
|
|
||||||
|
| Page | File | Changes |
|
||||||
|
|------|------|---------|
|
||||||
|
| Pricing | `marketing/pages/Pricing.tsx` | Already updated per session |
|
||||||
|
| Waitlist | `marketing/pages/Waitlist.tsx` | Update copy |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 4: Components to Update
|
||||||
|
|
||||||
|
### High Priority
|
||||||
|
|
||||||
|
| Component | File | Changes |
|
||||||
|
|-----------|------|---------|
|
||||||
|
| CreditBalanceWidget | `components/dashboard/CreditBalanceWidget.tsx` | Rename to ContentBalanceWidget |
|
||||||
|
| BillingBalancePanel | `components/billing/BillingBalancePanel.tsx` | Remove "Purchase Credits" link |
|
||||||
|
| BillingUsagePanel | `components/billing/BillingUsagePanel.tsx` | Simplify display |
|
||||||
|
| CurrentProcessingCard | `components/Automation/CurrentProcessingCard.tsx` | "Credits Used" → "Content Generated" |
|
||||||
|
| RunHistory | `components/Automation/RunHistory.tsx` | Same |
|
||||||
|
|
||||||
|
### Medium Priority (Can ship with these unchanged)
|
||||||
|
|
||||||
|
| Component | File | Notes |
|
||||||
|
|-----------|------|-------|
|
||||||
|
| CreditCostBreakdownPanel | `components/billing/CreditCostBreakdownPanel.tsx` | Admin only, keep |
|
||||||
|
| CreditCostsPanel | `components/billing/CreditCostsPanel.tsx` | Admin only, keep |
|
||||||
|
| UsageChartWidget | `components/dashboard/UsageChartWidget.tsx` | Keep for analytics |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 5: Routes to Update
|
||||||
|
|
||||||
|
### File: `frontend/src/App.tsx`
|
||||||
|
|
||||||
|
**Remove from user routes:**
|
||||||
|
```typescript
|
||||||
|
// Remove or hide:
|
||||||
|
<Route path="/account/purchase-credits" element={<PurchaseCreditsPage />} />
|
||||||
|
<Route path="/billing/credits" element={<Credits />} />
|
||||||
|
```
|
||||||
|
|
||||||
|
**Keep for admin access only** (add admin check)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 6: Terminology Mapping
|
||||||
|
|
||||||
|
Use this mapping consistently:
|
||||||
|
|
||||||
|
| OLD (Backend/Internal) | NEW (User-Facing) |
|
||||||
|
|------------------------|-------------------|
|
||||||
|
| credits | content pieces |
|
||||||
|
| credit balance | content remaining |
|
||||||
|
| credits used | content generated |
|
||||||
|
| plan_credits_per_month | monthly content limit |
|
||||||
|
| credits_remaining | content pieces left |
|
||||||
|
| insufficient credits | content limit reached |
|
||||||
|
| purchase credits | upgrade plan |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 7: Error Messages
|
||||||
|
|
||||||
|
### File: Create `frontend/src/utils/userFriendlyMessages.ts`
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
export const USER_MESSAGES = {
|
||||||
|
INSUFFICIENT_CREDITS: "You've reached your content limit for this month. Upgrade your plan for more.",
|
||||||
|
CREDIT_DEDUCTED: "Content piece generated successfully",
|
||||||
|
LOW_BALANCE: "You're running low on content pieces this month",
|
||||||
|
|
||||||
|
// Operation-specific
|
||||||
|
CLUSTERING_SUCCESS: "Keywords organized into topics",
|
||||||
|
IDEAS_SUCCESS: "Content ideas generated",
|
||||||
|
CONTENT_SUCCESS: "Article draft created",
|
||||||
|
IMAGES_SUCCESS: "Images generated",
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Implementation Order
|
||||||
|
|
||||||
|
### Day 1: Core Changes (Ship Blocker)
|
||||||
|
|
||||||
|
1. [ ] Update `billingStore.ts` with computed properties
|
||||||
|
2. [ ] Update `AppLayout.tsx` header metric
|
||||||
|
3. [ ] Update `PlansAndBillingPage.tsx` - remove Credits tab
|
||||||
|
4. [ ] Hide Purchase Credits route (don't delete)
|
||||||
|
|
||||||
|
### Day 2: Dashboard & Key Pages
|
||||||
|
|
||||||
|
5. [ ] Update `Dashboard/Home.tsx` labels
|
||||||
|
6. [ ] Update `UsageAnalyticsPage.tsx` labels
|
||||||
|
7. [ ] Update `AutomationPage.tsx` labels
|
||||||
|
8. [ ] Update `Help.tsx` FAQ content
|
||||||
|
|
||||||
|
### Day 3: Components & Polish
|
||||||
|
|
||||||
|
9. [ ] Rename CreditBalanceWidget → ContentBalanceWidget
|
||||||
|
10. [ ] Update BillingBalancePanel
|
||||||
|
11. [ ] Update CurrentProcessingCard
|
||||||
|
12. [ ] Update RunHistory
|
||||||
|
|
||||||
|
### Day 4: Marketing & Final
|
||||||
|
|
||||||
|
13. [ ] Verify Pricing page is correct
|
||||||
|
14. [ ] Update Waitlist/Tour pages
|
||||||
|
15. [ ] Update SignUp form text
|
||||||
|
16. [ ] Final QA pass
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What NOT to Change
|
||||||
|
|
||||||
|
Keep these exactly as-is:
|
||||||
|
|
||||||
|
1. **Backend API endpoints** - All `/v1/billing/*` endpoints
|
||||||
|
2. **Database models** - CreditBalance, CreditUsage, etc.
|
||||||
|
3. **Credit deduction logic** - In business services
|
||||||
|
4. **Admin pages** - Keep full credit visibility for admins
|
||||||
|
5. **API responses** - Backend still returns `credits`, frontend translates
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Backend Soft Limits (Already Implemented)
|
||||||
|
|
||||||
|
Per the pricing session, backend should enforce:
|
||||||
|
|
||||||
|
| Limit | Starter | Growth | Scale |
|
||||||
|
|-------|---------|--------|-------|
|
||||||
|
| Content pieces/mo | 50 | 200 | 500 |
|
||||||
|
| Keyword imports/mo | 500 | 2,000 | 5,000 |
|
||||||
|
| Clustering ops/mo | 100 | 400 | 1,000 |
|
||||||
|
| Idea generations/mo | 150 | 600 | 1,500 |
|
||||||
|
|
||||||
|
**If not implemented:** These are hidden from user but prevent abuse. Add backend validation in:
|
||||||
|
- `backend/igny8_core/business/billing/services.py`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Wins (Can Do Now)
|
||||||
|
|
||||||
|
### 1. Header Metric (5 min)
|
||||||
|
|
||||||
|
In `AppLayout.tsx`, change:
|
||||||
|
```typescript
|
||||||
|
label: 'Credits',
|
||||||
|
```
|
||||||
|
to:
|
||||||
|
```typescript
|
||||||
|
label: 'Content',
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Help FAQ (5 min)
|
||||||
|
|
||||||
|
In `Help.tsx`, update the credits question to:
|
||||||
|
```
|
||||||
|
question: "How does content usage work?"
|
||||||
|
answer: "Each plan includes a monthly content allowance. Creating articles, generating ideas, and producing images all count toward your monthly limit. View your usage in Settings > Usage."
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Hide Purchase Credits Link (5 min)
|
||||||
|
|
||||||
|
In `BillingBalancePanel.tsx`, remove or conditionally hide:
|
||||||
|
```typescript
|
||||||
|
<Link to="/account/purchase-credits">Purchase Credits</Link>
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing Checklist
|
||||||
|
|
||||||
|
Before production:
|
||||||
|
|
||||||
|
- [ ] User can see "47/50 Content" in header (not "835 Credits")
|
||||||
|
- [ ] Plans page shows content pieces, not credits
|
||||||
|
- [ ] No "Purchase Credits" visible to regular users
|
||||||
|
- [ ] Automation shows "Content Generated: 5" not "Credits Used: 5"
|
||||||
|
- [ ] Error on limit shows "Content limit reached" not "Insufficient credits"
|
||||||
|
- [ ] Dashboard shows simple progress bar with content count
|
||||||
|
- [ ] Help/FAQ explains content pieces, not credits
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
**Total effort:** ~2-3 days of frontend work
|
||||||
|
**Backend changes:** Zero
|
||||||
|
**Risk:** Low (display-only changes)
|
||||||
|
**Rollback:** Easy (revert frontend only)
|
||||||
|
|
||||||
|
The system still tracks everything internally with credits. Users just see a simpler "content pieces" model that maps 1:1 to content credits.
|
||||||
35
README.md
35
README.md
@@ -1,12 +1,21 @@
|
|||||||
# IGNY8 - AI-Powered SEO Content Platform
|
# IGNY8 - AI-Powered SEO Content Platform
|
||||||
|
|
||||||
**Version:** 1.0.0
|
**Version:** 1.0.5
|
||||||
**License:** Proprietary
|
**License:** Proprietary
|
||||||
**Website:** https://igny8.com
|
**Website:** https://igny8.com
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Links
|
||||||
|
|
||||||
|
| Document | Description |
|
||||||
|
|----------|-------------|
|
||||||
|
| [IGNY8-APP.md](IGNY8-APP.md) | Executive summary (non-technical) |
|
||||||
|
| [docs/INDEX.md](docs/INDEX.md) | Full documentation index |
|
||||||
|
| [CHANGELOG.md](CHANGELOG.md) | Version history |
|
||||||
|
| [RULES.md](RULES.md) | Documentation maintenance rules |
|
||||||
|
|
||||||
---
|
---
|
||||||
git push test 1
|
|
||||||
|
|
||||||
## What is IGNY8?
|
## What is IGNY8?
|
||||||
|
|
||||||
@@ -17,8 +26,8 @@ IGNY8 is a full-stack SaaS platform that combines AI-powered content generation
|
|||||||
- 🔍 **Smart Keyword Management** - Import, cluster, and organize keywords with AI
|
- 🔍 **Smart Keyword Management** - Import, cluster, and organize keywords with AI
|
||||||
- ✍️ **AI Content Generation** - Generate SEO-optimized blog posts using GPT-4
|
- ✍️ **AI Content Generation** - Generate SEO-optimized blog posts using GPT-4
|
||||||
- 🖼️ **AI Image Creation** - Auto-generate featured and in-article images
|
- 🖼️ **AI Image Creation** - Auto-generate featured and in-article images
|
||||||
- 🔗 **Internal Linking** - AI-powered link suggestions for SEO
|
- 🔗 **Internal Linking** - AI-powered link suggestions (coming soon)
|
||||||
- 📊 **Content Optimization** - Analyze and score content quality
|
- 📊 **Content Optimization** - Analyze and score content quality (coming soon)
|
||||||
- 🔄 **WordPress Integration** - Bidirectional sync with WordPress sites
|
- 🔄 **WordPress Integration** - Bidirectional sync with WordPress sites
|
||||||
- 📈 **Usage-Based Billing** - Credit system for AI operations
|
- 📈 **Usage-Based Billing** - Credit system for AI operations
|
||||||
- 👥 **Multi-Tenancy** - Manage multiple sites and teams
|
- 👥 **Multi-Tenancy** - Manage multiple sites and teams
|
||||||
@@ -27,14 +36,24 @@ IGNY8 is a full-stack SaaS platform that combines AI-powered content generation
|
|||||||
|
|
||||||
## Repository Structure
|
## Repository Structure
|
||||||
|
|
||||||
This monorepo contains two main applications and documentation:
|
|
||||||
|
|
||||||
```
|
```
|
||||||
igny8/
|
igny8/
|
||||||
|
├── README.md # This file
|
||||||
|
├── CHANGELOG.md # Version history
|
||||||
|
├── IGNY8-APP.md # Executive summary
|
||||||
|
├── RULES.md # Documentation rules
|
||||||
├── backend/ # Django REST API + Celery
|
├── backend/ # Django REST API + Celery
|
||||||
├── frontend/ # React + Vite SPA
|
├── frontend/ # React + Vite SPA
|
||||||
├── docs/ # Documentation index and topic folders
|
├── docs/ # Full documentation
|
||||||
└── docker-compose.app.yml # Docker deployment config
|
│ ├── INDEX.md # Documentation navigation
|
||||||
|
│ ├── 00-SYSTEM/ # Architecture & auth
|
||||||
|
│ ├── 10-MODULES/ # Module documentation
|
||||||
|
│ ├── 20-API/ # API endpoints
|
||||||
|
│ ├── 30-FRONTEND/ # Frontend pages & stores
|
||||||
|
│ ├── 40-WORKFLOWS/ # Cross-module workflows
|
||||||
|
│ ├── 50-DEPLOYMENT/ # Deployment guides
|
||||||
|
│ └── 90-REFERENCE/ # Models & AI functions
|
||||||
|
└── docker-compose.app.yml
|
||||||
```
|
```
|
||||||
|
|
||||||
**Separate Repository:**
|
**Separate Repository:**
|
||||||
|
|||||||
229
RULES.md
Normal file
229
RULES.md
Normal file
@@ -0,0 +1,229 @@
|
|||||||
|
# IGNY8 Documentation Rules
|
||||||
|
|
||||||
|
**Version:** 1.0
|
||||||
|
**Last Updated:** December 25, 2025
|
||||||
|
**Purpose:** Rules for maintaining consistent, accurate documentation
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Core Principles
|
||||||
|
|
||||||
|
1. **Single Source of Truth** - One file per topic, no duplicates
|
||||||
|
2. **Code is King** - Documentation reflects actual code, not plans
|
||||||
|
3. **Tables Over Prose** - Use tables for lists of 3+ items
|
||||||
|
4. **Code Paths Always** - Every feature links to exact files
|
||||||
|
5. **Brief But Complete** - No fluff, every word has purpose
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File Structure Rules
|
||||||
|
|
||||||
|
### Root Folder (/)
|
||||||
|
|
||||||
|
Only these MD files allowed in root:
|
||||||
|
- `CHANGELOG.md` - Version history
|
||||||
|
- `RULES.md` - This file
|
||||||
|
- `IGNY8-APP.md` - Executive summary (non-technical)
|
||||||
|
- `README.md` - Project quickstart
|
||||||
|
|
||||||
|
**DO NOT CREATE** random MD files in root. All documentation goes in `/docs/`.
|
||||||
|
|
||||||
|
### Docs Folder (/docs/)
|
||||||
|
|
||||||
|
```
|
||||||
|
docs/
|
||||||
|
├── INDEX.md # Master navigation (update when adding docs)
|
||||||
|
├── 00-SYSTEM/ # Architecture, auth, tenancy
|
||||||
|
├── 10-MODULES/ # One file per module
|
||||||
|
├── 20-API/ # API endpoints and schemas
|
||||||
|
├── 30-FRONTEND/ # Pages, stores, components
|
||||||
|
├── 40-WORKFLOWS/ # Cross-module flows
|
||||||
|
└── 90-REFERENCE/ # Models, AI functions, troubleshooting
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Document Templates
|
||||||
|
|
||||||
|
### Module Document
|
||||||
|
|
||||||
|
Every module doc must include:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# [Module Name]
|
||||||
|
|
||||||
|
**Last Verified:** YYYY-MM-DD
|
||||||
|
**Status:** ✅ Active | ⏸️ Inactive
|
||||||
|
**Backend Path:** `backend/...`
|
||||||
|
**Frontend Path:** `frontend/...`
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
| What | File | Key Items |
|
||||||
|
|------|------|-----------|
|
||||||
|
|
||||||
|
## Data Models
|
||||||
|
[Tables with fields]
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|
||||||
|
## Business Logic
|
||||||
|
[Key operations with credit costs]
|
||||||
|
|
||||||
|
## Frontend Pages
|
||||||
|
[Routes and components]
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
| Issue | Cause | Fix |
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
| Feature | Status | Description |
|
||||||
|
```
|
||||||
|
|
||||||
|
### Status Values
|
||||||
|
|
||||||
|
For Planned Changes tables:
|
||||||
|
- ✅ Completed
|
||||||
|
- 🐛 Bug (known issue)
|
||||||
|
- 🔜 Planned (confirmed future work)
|
||||||
|
- ⏸️ Pending (waiting on dependency)
|
||||||
|
- ❌ Cancelled
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Update Rules
|
||||||
|
|
||||||
|
### When to Update Documentation
|
||||||
|
|
||||||
|
| Event | Action |
|
||||||
|
|-------|--------|
|
||||||
|
| New feature added | Update relevant module doc |
|
||||||
|
| Bug fixed | Add to Common Issues if recurring |
|
||||||
|
| API endpoint changed | Update API endpoints table |
|
||||||
|
| Model changed | Update data models section |
|
||||||
|
| Config changed | Update relevant doc |
|
||||||
|
| Release | Update CHANGELOG.md |
|
||||||
|
|
||||||
|
### What to Update
|
||||||
|
|
||||||
|
| Change Type | Files to Update |
|
||||||
|
|-------------|-----------------|
|
||||||
|
| New module | Create module doc, update INDEX.md |
|
||||||
|
| New endpoint | Module doc + API/ENDPOINTS.md |
|
||||||
|
| New model | Module doc + 90-REFERENCE/MODELS.md |
|
||||||
|
| Frontend page | Module doc + 30-FRONTEND/PAGES.md |
|
||||||
|
| Bug fix | Common Issues in relevant doc |
|
||||||
|
|
||||||
|
### "Last Verified" Dates
|
||||||
|
|
||||||
|
Update `Last Verified:` date when:
|
||||||
|
- You confirm the doc matches current code
|
||||||
|
- After making changes to the doc
|
||||||
|
- During documentation reviews
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Writing Style
|
||||||
|
|
||||||
|
### Do
|
||||||
|
|
||||||
|
- Use present tense ("Creates" not "Will create")
|
||||||
|
- Use tables for any list of 3+ items
|
||||||
|
- Include exact file paths
|
||||||
|
- Link to related docs
|
||||||
|
- Keep sentences short
|
||||||
|
|
||||||
|
### Don't
|
||||||
|
|
||||||
|
- Don't explain obvious things
|
||||||
|
- Don't use jargon without defining
|
||||||
|
- Don't duplicate info across files
|
||||||
|
- Don't use passive voice
|
||||||
|
- Don't leave TODOs in docs (use Planned Changes table)
|
||||||
|
|
||||||
|
### Code Path Format
|
||||||
|
|
||||||
|
```
|
||||||
|
Backend: `backend/igny8_core/modules/planner/views.py:KeywordViewSet.bulk_delete`
|
||||||
|
Frontend: `frontend/src/pages/Planner/Keywords.tsx:handleBulkDelete`
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## AI Agent Instructions
|
||||||
|
|
||||||
|
When AI agents (Copilot, Claude, etc.) make changes:
|
||||||
|
|
||||||
|
### After Code Changes
|
||||||
|
|
||||||
|
1. Identify which module(s) were modified
|
||||||
|
2. Update the relevant module doc(s)
|
||||||
|
3. If API changed, update API endpoints
|
||||||
|
4. If model changed, update MODELS.md
|
||||||
|
5. Update CHANGELOG.md for significant changes
|
||||||
|
|
||||||
|
### What NOT to Do
|
||||||
|
|
||||||
|
- DO NOT create new MD files in root folder
|
||||||
|
- DO NOT create "summary" or "implementation" docs
|
||||||
|
- DO NOT duplicate existing documentation
|
||||||
|
- DO NOT add verbose explanations
|
||||||
|
- DO NOT leave placeholder content
|
||||||
|
|
||||||
|
### Verification Checklist
|
||||||
|
|
||||||
|
After documentation update:
|
||||||
|
- [ ] File is in correct location (/docs/ structure)
|
||||||
|
- [ ] INDEX.md updated if new file added
|
||||||
|
- [ ] No duplicate information
|
||||||
|
- [ ] All code paths are accurate
|
||||||
|
- [ ] Tables used appropriately
|
||||||
|
- [ ] Last Verified date updated
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Documentation Review Schedule
|
||||||
|
|
||||||
|
| Frequency | Action |
|
||||||
|
|-----------|--------|
|
||||||
|
| Per commit | Update affected docs |
|
||||||
|
| Weekly | Review recent changes vs docs |
|
||||||
|
| Monthly | Full audit: code vs docs |
|
||||||
|
| Per release | Update CHANGELOG.md, verify all docs |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Reference: File Locations
|
||||||
|
|
||||||
|
| Topic | File |
|
||||||
|
|-------|------|
|
||||||
|
| Master navigation | `docs/INDEX.md` |
|
||||||
|
| Architecture | `docs/00-SYSTEM/ARCHITECTURE.md` |
|
||||||
|
| Authentication | `docs/00-SYSTEM/AUTH-FLOWS.md` |
|
||||||
|
| Multi-tenancy | `docs/00-SYSTEM/TENANCY.md` |
|
||||||
|
| Planner module | `docs/10-MODULES/PLANNER.md` |
|
||||||
|
| Writer module | `docs/10-MODULES/WRITER.md` |
|
||||||
|
| Billing module | `docs/10-MODULES/BILLING.md` |
|
||||||
|
| Automation | `docs/10-MODULES/AUTOMATION.md` |
|
||||||
|
| Integrations | `docs/10-MODULES/INTEGRATIONS.md` |
|
||||||
|
| Settings | `docs/10-MODULES/SYSTEM-SETTINGS.md` |
|
||||||
|
| Linker (inactive) | `docs/10-MODULES/LINKER.md` |
|
||||||
|
| Optimizer (inactive) | `docs/10-MODULES/OPTIMIZER.md` |
|
||||||
|
| Publisher | `docs/10-MODULES/PUBLISHER.md` |
|
||||||
|
| API endpoints | `docs/20-API/ENDPOINTS.md` |
|
||||||
|
| All models | `docs/90-REFERENCE/MODELS.md` |
|
||||||
|
| Troubleshooting | `docs/90-REFERENCE/TROUBLESHOOTING.md` |
|
||||||
|
| Version history | `CHANGELOG.md` |
|
||||||
|
| Executive summary | `IGNY8-APP.md` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Enforcement
|
||||||
|
|
||||||
|
These rules are enforced by:
|
||||||
|
1. Code review checklist includes documentation
|
||||||
|
2. AI agents instructed to follow these rules
|
||||||
|
3. Monthly documentation audits
|
||||||
|
4. CHANGELOG must be updated for releases
|
||||||
|
|
||||||
|
**When in doubt: Update the existing doc, don't create a new file.**
|
||||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
@@ -1,79 +0,0 @@
|
|||||||
# System Architecture Overview
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe how IGNY8 is structured across backend, frontend, and integrations, grounded in the current codebase. Covers core services, middleware, and platform composition.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Backend project root: `backend/igny8_core/`
|
|
||||||
- Settings and service wiring: `backend/igny8_core/settings.py`
|
|
||||||
- URL routing: `backend/igny8_core/urls.py`
|
|
||||||
- Middleware: `backend/igny8_core/middleware/request_id.py`, `backend/igny8_core/middleware/resource_tracker.py`, `backend/igny8_core/auth/middleware.py`
|
|
||||||
- Auth models and tenancy bases: `backend/igny8_core/auth/models.py`
|
|
||||||
- DRF base behaviors: `backend/igny8_core/api/base.py`
|
|
||||||
- Custom auth classes: `backend/igny8_core/api/authentication.py`
|
|
||||||
- Frontend SPA: `frontend/` (Vite + React; dependencies in `frontend/package.json`)
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Django/DRF backend providing multi-tenant APIs for planner, writer, system, billing, automation, linker, optimizer, publisher, and integration modules.
|
|
||||||
- Middleware adds per-request IDs, tenant context, and optional resource tracking for admin diagnostics.
|
|
||||||
- REST API routing under `/api/v1/*` with unified response/error handling and scoped throttling.
|
|
||||||
- Celery-backed async work (configured in settings) for AI, automation, and publishing.
|
|
||||||
- React/Vite frontend consuming the API; authentication via JWT or session; API key support for WordPress bridge.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Backend apps registered in `INSTALLED_APPS` include auth, AI framework, planner, writer, system, billing, automation, optimization, publishing, integration, linker, optimizer, and publisher modules; these are loaded via Django app configs in `settings.py`.
|
|
||||||
- Middleware order enforces security, CORS, session, CSRF, Django auth, then custom layers: request ID, account context, resource tracking, messages, and clickjacking protection.
|
|
||||||
- URL map (`urls.py`) exposes admin, CSV admin utilities for industries/seed keywords, and module routers for auth, account, planner, writer, system, billing (user + admin), automation, linker, optimizer, publisher, and integration. OpenAPI docs available at `/api/docs` and `/api/redoc`.
|
|
||||||
- REST framework defaults use custom pagination, filtering, search, ordering, and a custom exception handler (enabled unless `IGNY8_USE_UNIFIED_EXCEPTION_HANDLER` is false). Authentication stack orders API key, JWT, CSRF-exempt session, then basic auth. Throttle scopes are predefined per domain (AI, content, auth, planner, writer, system, billing, linker, optimizer, integration).
|
|
||||||
- CORS allows the IGNY8 domains plus local development hosts; credentials and specific debug headers are permitted, and request/resource tracking IDs are exposed in response headers.
|
|
||||||
- Celery is configured to use Redis by default for broker and result backend, with JSON serialization, task time limits, and single-prefetch workers.
|
|
||||||
- Logging writes to console and rotating files for publish/sync, WordPress API calls, and webhooks, with request IDs available via middleware.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code, just explanation)
|
|
||||||
- Multi-tenancy base classes (`AccountBaseModel`, `SiteSectorBaseModel`) add `account`, `site`, and `sector` scoping plus validation; defined in `auth/models.py`.
|
|
||||||
- Core tenancy entities: `Account`, `Plan`, `Subscription`, `Site`, `Sector`, `Industry`, `IndustrySector`, `SeedKeyword`, `SiteUserAccess`, `User`, and `PasswordResetToken` in `auth/models.py`.
|
|
||||||
- These bases are consumed by downstream modules (planner, writer, billing, automation, etc.) to enforce tenant, site, and sector ownership.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Incoming HTTP requests enter Django middleware: security → WhiteNoise → CORS → session → common/CSRF → Django auth → `RequestIDMiddleware` (assigns `X-Request-ID`) → `AccountContextMiddleware` (sets `request.account` via session or JWT) → `ResourceTrackingMiddleware` (when enabled for admins) → messages → clickjacking.
|
|
||||||
- DRF viewsets inherit from base classes in `api/base.py` to auto-filter querysets by account (and site/sector where applicable) and emit unified responses.
|
|
||||||
- URL dispatch routes to module-specific routers under `/api/v1/*`. CSV admin helpers are mounted before admin to avoid routing conflicts.
|
|
||||||
- Background tasks are dispatched via Celery using Redis endpoints from `settings.py`.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Auth middleware sets `request.account` consumed by `AccountModelViewSet` and `SiteSectorModelViewSet` to filter data.
|
|
||||||
- API key auth (WordPress bridge) sets both `request.account` and `request.site` for integration endpoints.
|
|
||||||
- Resource tracking middleware exposes metrics via cache keyed by the request-specific tracking ID added to responses.
|
|
||||||
- OpenAPI generation (drf-spectacular) documents all modules and respects unified response schemas.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Request lifecycle: ID assignment → tenant resolution → optional resource profiling → viewset execution → unified response with optional pagination and request/resource IDs in headers.
|
|
||||||
- Tenancy lifecycle: Account/Plan/Subscription fields in models track status and retention; soft delete support on models that implement it.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Global DRF exception handler (custom when enabled) wraps errors into unified JSON with `success=false`.
|
|
||||||
- Account middleware denies access when account or plan is missing/inactive, returning structured JSON and logging out session users.
|
|
||||||
- Viewset overrides in `api/base.py` return unified error payloads for validation and 404/500 cases.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Account is injected via middleware (session or JWT). Base viewsets filter querysets by `account`, unless the user is admin/developer/system account (override path).
|
|
||||||
- `SiteSectorModelViewSet` adds site/sector filtering when models carry those fields; validation ensures site/sector belong to the same account.
|
|
||||||
- Role helpers (`User.is_admin_or_developer`, `User.is_system_account_user`) allow bypass for privileged users; otherwise, data is restricted to the resolved account (and site/sector for derived models).
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Plan and account billing fields live in `auth/models.py` (`Plan.included_credits`, `Account.credits`, Stripe IDs). Status enforcement occurs in `AccountContextMiddleware` by requiring an active plan; billing modules implement credit logic (covered in backend/billing docs).
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Celery configuration in `settings.py` sets Redis broker/backend, JSON serialization, time limits, and worker tuning. Task modules (AI, automation, publishing) use this setup; scheduling uses Celery Beat state.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Middleware-first tenant resolution ensures consistent scoping before view logic.
|
|
||||||
- Admin/developer/system-account overrides allow cross-tenant operations for ops while protecting system accounts from deletion.
|
|
||||||
- Unified API responses and throttling scopes enforce consistent client behavior and rate safety.
|
|
||||||
- Redis-backed Celery keeps async workloads out of request path with strict time limits.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Add new apps to `INSTALLED_APPS` and mount URLs under `/api/v1/*` in `urls.py`.
|
|
||||||
- Inherit from `AccountModelViewSet` or `SiteSectorModelViewSet` to automatically enforce tenant/site/sector scoping and unified responses.
|
|
||||||
- Use existing middleware; do not reorder request ID or account context layers, as downstream views rely on them.
|
|
||||||
- Configure environment via `settings.py` variables (DB, JWT, Celery, CORS, Stripe/PayPal) rather than hardcoding values.
|
|
||||||
289
docs/00-SYSTEM/ARCHITECTURE.md
Normal file
289
docs/00-SYSTEM/ARCHITECTURE.md
Normal file
@@ -0,0 +1,289 @@
|
|||||||
|
# System Architecture
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Backend Path:** `backend/igny8_core/`
|
||||||
|
**Frontend Path:** `frontend/src/`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Tech Stack
|
||||||
|
|
||||||
|
| Layer | Technology | Version | Purpose |
|
||||||
|
|-------|------------|---------|---------|
|
||||||
|
| **Backend Framework** | Django | 5.1 | Web framework |
|
||||||
|
| **API Layer** | Django REST Framework | 3.15 | REST API |
|
||||||
|
| **Database** | PostgreSQL | 16 | Primary data store |
|
||||||
|
| **Cache/Queue** | Redis | 7 | Caching, Celery broker |
|
||||||
|
| **Task Queue** | Celery | 5.4 | Async task processing |
|
||||||
|
| **Frontend Framework** | React | 18 | UI framework |
|
||||||
|
| **Build Tool** | Vite | 5 | Frontend bundler |
|
||||||
|
| **Styling** | Tailwind CSS | 3 | Utility CSS |
|
||||||
|
| **State Management** | Zustand | 4 | React state |
|
||||||
|
| **Language** | TypeScript | 5 | Frontend typing |
|
||||||
|
| **Web Server** | Caddy | 2 | Reverse proxy, SSL |
|
||||||
|
| **Containerization** | Docker | - | Deployment |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## High-Level Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────────┐
|
||||||
|
│ CLIENTS │
|
||||||
|
│ React SPA (app.igny8.com) │ WordPress Plugin │ Admin │
|
||||||
|
└─────────────────────────────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────────────────────────────────────────────────────┐
|
||||||
|
│ CADDY (Reverse Proxy) │
|
||||||
|
│ SSL termination, routing, static files │
|
||||||
|
└─────────────────────────────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────────────────────────────────────────────────────┐
|
||||||
|
│ DJANGO REST FRAMEWORK │
|
||||||
|
│ │
|
||||||
|
│ Middleware Stack: │
|
||||||
|
│ SecurityMiddleware → WhiteNoise → CORS → Session → CSRF → │
|
||||||
|
│ DjangoAuth → RequestIDMiddleware → AccountContextMiddleware → │
|
||||||
|
│ ResourceTrackingMiddleware │
|
||||||
|
│ │
|
||||||
|
│ API: /api/v1/* → ViewSets → Services → Models │
|
||||||
|
└─────────────────────────────────────────────────────────────────┘
|
||||||
|
│ │
|
||||||
|
▼ ▼
|
||||||
|
┌─────────────────────┐ ┌─────────────────────────────────────┐
|
||||||
|
│ PostgreSQL │ │ Redis │
|
||||||
|
│ Primary Database │ │ Sessions, Cache, Celery Broker │
|
||||||
|
└─────────────────────┘ └─────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────────────────────────────────────────────────────┐
|
||||||
|
│ CELERY WORKERS │
|
||||||
|
│ AI Tasks, Automation, Publishing, Background Jobs │
|
||||||
|
└─────────────────────────────────────────────────────────────────┘
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────────────────────────────────────────────────────┐
|
||||||
|
│ EXTERNAL SERVICES │
|
||||||
|
│ OpenAI API (GPT-4, DALL-E) │ Runware │ WordPress Sites │
|
||||||
|
└─────────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Backend Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
backend/igny8_core/
|
||||||
|
├── settings.py # Django settings, all config
|
||||||
|
├── urls.py # Root URL routing
|
||||||
|
├── celery.py # Celery configuration
|
||||||
|
├── wsgi.py / asgi.py # WSGI/ASGI entry points
|
||||||
|
│
|
||||||
|
├── auth/ # Authentication & tenancy
|
||||||
|
│ ├── models.py # User, Account, Site, Sector, Plan
|
||||||
|
│ ├── views.py # Login, register, password reset
|
||||||
|
│ ├── middleware.py # AccountContextMiddleware
|
||||||
|
│ └── urls.py # /api/v1/auth/*
|
||||||
|
│
|
||||||
|
├── api/ # API infrastructure
|
||||||
|
│ ├── base.py # AccountModelViewSet, SiteSectorModelViewSet
|
||||||
|
│ ├── authentication.py # JWT, API key auth classes
|
||||||
|
│ └── pagination.py # Unified pagination
|
||||||
|
│
|
||||||
|
├── ai/ # AI engine
|
||||||
|
│ ├── engine.py # AIEngine orchestrator
|
||||||
|
│ ├── functions/ # AutoCluster, GenerateIdeas, GenerateContent, etc.
|
||||||
|
│ ├── registry.py # Function registry
|
||||||
|
│ └── progress.py # Progress tracking
|
||||||
|
│
|
||||||
|
├── modules/ # Feature modules (API layer)
|
||||||
|
│ ├── planner/ # Keywords, Clusters, Ideas
|
||||||
|
│ ├── writer/ # Tasks, Content, Images
|
||||||
|
│ ├── billing/ # Credits, usage, transactions
|
||||||
|
│ ├── integration/ # WordPress integration
|
||||||
|
│ ├── system/ # Settings, prompts, AI config
|
||||||
|
│ ├── linker/ # Internal linking (inactive)
|
||||||
|
│ ├── optimizer/ # Content optimization (inactive)
|
||||||
|
│ └── publisher/ # Publishing pipeline
|
||||||
|
│
|
||||||
|
├── business/ # Business logic (services)
|
||||||
|
│ ├── automation/ # 7-stage automation pipeline
|
||||||
|
│ ├── billing/ # Credit service, payment processing
|
||||||
|
│ ├── content/ # Content generation orchestration
|
||||||
|
│ ├── integration/ # Sync services
|
||||||
|
│ ├── linking/ # Link processing
|
||||||
|
│ ├── optimization/ # Content optimization
|
||||||
|
│ ├── planning/ # Clustering, idea generation
|
||||||
|
│ └── publishing/ # Publishing orchestration
|
||||||
|
│
|
||||||
|
├── middleware/ # Custom middleware
|
||||||
|
│ ├── request_id.py # X-Request-ID header
|
||||||
|
│ └── resource_tracker.py # Resource tracking for admins
|
||||||
|
│
|
||||||
|
└── tasks/ # Celery tasks
|
||||||
|
└── *.py # Background job definitions
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Frontend Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
frontend/src/
|
||||||
|
├── main.tsx # Entry point
|
||||||
|
├── App.tsx # Root component, routing
|
||||||
|
├── index.css # Global styles, Tailwind
|
||||||
|
│
|
||||||
|
├── api/ # API clients
|
||||||
|
│ ├── linker.api.ts
|
||||||
|
│ ├── optimizer.api.ts
|
||||||
|
│ └── ...
|
||||||
|
│
|
||||||
|
├── services/
|
||||||
|
│ └── api.ts # Main API service (2500+ lines)
|
||||||
|
│
|
||||||
|
├── store/ # Zustand stores
|
||||||
|
│ ├── authStore.ts # Authentication state
|
||||||
|
│ ├── siteStore.ts # Active site
|
||||||
|
│ ├── sectorStore.ts # Active sector
|
||||||
|
│ ├── billingStore.ts # Billing state
|
||||||
|
│ ├── moduleStore.ts # Module enable/disable
|
||||||
|
│ └── ...
|
||||||
|
│
|
||||||
|
├── pages/ # Route pages
|
||||||
|
│ ├── Dashboard/
|
||||||
|
│ ├── Planner/
|
||||||
|
│ ├── Writer/
|
||||||
|
│ ├── Automation/
|
||||||
|
│ ├── Linker/
|
||||||
|
│ ├── Optimizer/
|
||||||
|
│ ├── Settings/
|
||||||
|
│ ├── Billing/
|
||||||
|
│ └── Auth/
|
||||||
|
│
|
||||||
|
├── components/ # Reusable components
|
||||||
|
│ ├── ProgressModal.tsx # AI progress display
|
||||||
|
│ ├── ConfirmDialog.tsx
|
||||||
|
│ └── ...
|
||||||
|
│
|
||||||
|
├── layout/ # Layout components
|
||||||
|
│ ├── AppLayout.tsx
|
||||||
|
│ └── AppSidebar.tsx
|
||||||
|
│
|
||||||
|
├── hooks/ # Custom hooks
|
||||||
|
├── context/ # React contexts
|
||||||
|
└── utils/ # Utility functions
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Key Design Patterns
|
||||||
|
|
||||||
|
### 1. Multi-Tenant Data Isolation
|
||||||
|
|
||||||
|
All data is scoped to Account, with optional Site and Sector filtering:
|
||||||
|
|
||||||
|
```
|
||||||
|
Account (Tenant)
|
||||||
|
└── Site (e.g., myblog.com)
|
||||||
|
└── Sector (e.g., Technology, Health)
|
||||||
|
└── Keywords/Clusters/Ideas/Content
|
||||||
|
```
|
||||||
|
|
||||||
|
**Implementation:**
|
||||||
|
- `AccountBaseModel` - Base class for account-scoped models
|
||||||
|
- `SiteSectorBaseModel` - Base class for site/sector-scoped models
|
||||||
|
- `AccountModelViewSet` - Auto-filters queryset by request.account
|
||||||
|
- `SiteSectorModelViewSet` - Auto-filters by account + site + sector
|
||||||
|
|
||||||
|
### 2. Middleware-First Tenant Resolution
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Order in settings.py MIDDLEWARE
|
||||||
|
1. SecurityMiddleware
|
||||||
|
2. WhiteNoiseMiddleware
|
||||||
|
3. CorsMiddleware
|
||||||
|
4. SessionMiddleware
|
||||||
|
5. DjangoAuthenticationMiddleware
|
||||||
|
6. RequestIDMiddleware # Assigns X-Request-ID
|
||||||
|
7. AccountContextMiddleware # Sets request.account from session/JWT
|
||||||
|
8. ResourceTrackingMiddleware # Optional admin profiling
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Unified API Responses
|
||||||
|
|
||||||
|
All API responses follow this structure:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"success": true,
|
||||||
|
"data": { ... },
|
||||||
|
"message": "Optional message"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Error responses:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"success": false,
|
||||||
|
"error": "Error message",
|
||||||
|
"code": "ERROR_CODE"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Credit-Based Operations
|
||||||
|
|
||||||
|
All AI operations check and deduct credits:
|
||||||
|
1. Pre-check: `CreditService.check_credits()`
|
||||||
|
2. Execute: AI function runs
|
||||||
|
3. Post-deduct: `CreditService.deduct_credits_for_operation()`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Environment Configuration
|
||||||
|
|
||||||
|
Key environment variables (from `settings.py`):
|
||||||
|
|
||||||
|
| Variable | Purpose |
|
||||||
|
|----------|---------|
|
||||||
|
| `DATABASE_URL` | PostgreSQL connection |
|
||||||
|
| `REDIS_URL` | Redis connection |
|
||||||
|
| `SECRET_KEY` | Django secret |
|
||||||
|
| `JWT_SECRET_KEY` | JWT signing key |
|
||||||
|
| `CORS_ALLOWED_ORIGINS` | Allowed frontend origins |
|
||||||
|
| `CELERY_BROKER_URL` | Celery broker (Redis) |
|
||||||
|
|
||||||
|
AI keys are stored in `GlobalIntegrationSettings` (database), not env vars.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Deployment Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────────┐
|
||||||
|
│ Docker Compose │
|
||||||
|
│ │
|
||||||
|
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
|
||||||
|
│ │ Frontend │ │ Backend │ │ Worker │ │
|
||||||
|
│ │ (Caddy) │ │ (Django) │ │ (Celery) │ │
|
||||||
|
│ └─────────────┘ └─────────────┘ └─────────────┘ │
|
||||||
|
│ │ │ │ │
|
||||||
|
│ └────────────────┼────────────────┘ │
|
||||||
|
│ │ │
|
||||||
|
│ ┌─────────────┐ ┌─────────────┐ │
|
||||||
|
│ │ PostgreSQL │ │ Redis │ │
|
||||||
|
│ └─────────────┘ └─────────────┘ │
|
||||||
|
└─────────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
|
||||||
|
| Feature | Status | Description |
|
||||||
|
|---------|--------|-------------|
|
||||||
|
| AIModelConfig Database | 🔜 Planned | Move AI model pricing from constants to database |
|
||||||
|
| Module Guard Extension | 🔜 Planned | Extend linker/optimizer disable to all pages (currently sidebar only) |
|
||||||
|
| Multi-provider AI | 🔜 Planned | Support for Anthropic, Google AI |
|
||||||
254
docs/00-SYSTEM/AUTH-FLOWS.md
Normal file
254
docs/00-SYSTEM/AUTH-FLOWS.md
Normal file
@@ -0,0 +1,254 @@
|
|||||||
|
# Authentication & Authorization
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Backend Path:** `backend/igny8_core/auth/`
|
||||||
|
**Frontend Path:** `frontend/src/store/authStore.ts`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
|
||||||
|
| What | File | Key Functions |
|
||||||
|
|------|------|---------------|
|
||||||
|
| User Model | `auth/models.py` | `User`, `Account`, `Plan` |
|
||||||
|
| Auth Views | `auth/views.py` | `LoginView`, `RegisterView`, `RefreshTokenView` |
|
||||||
|
| Middleware | `auth/middleware.py` | `AccountContextMiddleware` |
|
||||||
|
| JWT Auth | `api/authentication.py` | `JWTAuthentication`, `CookieJWTAuthentication` |
|
||||||
|
| API Key Auth | `api/authentication.py` | `APIKeyAuthentication` |
|
||||||
|
| Frontend Store | `store/authStore.ts` | `useAuthStore` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Authentication Methods
|
||||||
|
|
||||||
|
### 1. JWT Token Authentication (Primary)
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. User logs in via `/api/v1/auth/login/`
|
||||||
|
2. Backend returns `access_token` (15 min) + `refresh_token` (7 days)
|
||||||
|
3. Frontend stores tokens in localStorage and Zustand store
|
||||||
|
4. All API requests include `Authorization: Bearer <access_token>`
|
||||||
|
5. Token refresh via `/api/v1/auth/token/refresh/`
|
||||||
|
|
||||||
|
**Token Payload:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"user_id": 123,
|
||||||
|
"account_id": 456,
|
||||||
|
"email": "user@example.com",
|
||||||
|
"exp": 1735123456,
|
||||||
|
"iat": 1735122456
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Session Authentication (Admin/Fallback)
|
||||||
|
|
||||||
|
- Used by Django Admin interface
|
||||||
|
- Cookie-based session with CSRF protection
|
||||||
|
- Redis-backed sessions (prevents user swapping bug)
|
||||||
|
|
||||||
|
### 3. API Key Authentication (WordPress Bridge)
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. Account generates API key in settings
|
||||||
|
2. WordPress plugin uses `Authorization: ApiKey <key>`
|
||||||
|
3. Backend validates key, sets `request.account` and `request.site`
|
||||||
|
|
||||||
|
**Use Cases:**
|
||||||
|
- WordPress content sync
|
||||||
|
- External integrations
|
||||||
|
- Headless CMS connections
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| POST | `/api/v1/auth/register/` | `RegisterView` | Create new user + account |
|
||||||
|
| POST | `/api/v1/auth/login/` | `LoginView` | Authenticate, return tokens |
|
||||||
|
| POST | `/api/v1/auth/logout/` | `LogoutView` | Invalidate tokens |
|
||||||
|
| POST | `/api/v1/auth/token/refresh/` | `RefreshTokenView` | Refresh access token |
|
||||||
|
| POST | `/api/v1/auth/password/change/` | `ChangePasswordView` | Change password |
|
||||||
|
| POST | `/api/v1/auth/password/reset/` | `RequestPasswordResetView` | Request reset email |
|
||||||
|
| POST | `/api/v1/auth/password/reset/confirm/` | `ResetPasswordView` | Confirm reset with token |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## User Roles
|
||||||
|
|
||||||
|
| Role | Code | Permissions |
|
||||||
|
|------|------|-------------|
|
||||||
|
| **Developer** | `developer` | Full access across ALL accounts (superuser) |
|
||||||
|
| **Admin** | `admin` | Full access to own account |
|
||||||
|
| **Manager** | `manager` | Manage content, view billing |
|
||||||
|
| **Editor** | `editor` | AI content, manage clusters/tasks |
|
||||||
|
| **Viewer** | `viewer` | Read-only dashboards |
|
||||||
|
| **System Bot** | `system_bot` | System automation (internal) |
|
||||||
|
|
||||||
|
**Role Hierarchy:**
|
||||||
|
```
|
||||||
|
developer > admin > manager > editor > viewer
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Middleware: AccountContextMiddleware
|
||||||
|
|
||||||
|
**File:** `auth/middleware.py`
|
||||||
|
|
||||||
|
**Purpose:** Injects `request.account` on every request
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. Check for JWT token → extract account_id
|
||||||
|
2. Check for session → get account from session
|
||||||
|
3. Check for API key → get account from key
|
||||||
|
4. Validate account exists and is active
|
||||||
|
5. Validate plan exists and is active
|
||||||
|
6. Set `request.account`, `request.user`
|
||||||
|
|
||||||
|
**Error Responses:**
|
||||||
|
- No account: 403 with JSON error
|
||||||
|
- Inactive plan: 402 with JSON error
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Frontend Auth Store
|
||||||
|
|
||||||
|
**File:** `store/authStore.ts`
|
||||||
|
|
||||||
|
**State:**
|
||||||
|
```typescript
|
||||||
|
{
|
||||||
|
user: User | null;
|
||||||
|
token: string | null;
|
||||||
|
refreshToken: string | null;
|
||||||
|
isAuthenticated: boolean;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Actions:**
|
||||||
|
- `login(email, password)` - Authenticate and store tokens
|
||||||
|
- `register(data)` - Create account and store tokens
|
||||||
|
- `logout()` - Clear tokens and reset stores
|
||||||
|
- `refreshToken()` - Refresh access token
|
||||||
|
- `checkAuth()` - Verify current auth state
|
||||||
|
|
||||||
|
**Critical Implementation:**
|
||||||
|
```typescript
|
||||||
|
// Tokens are written synchronously to localStorage
|
||||||
|
// This prevents race conditions where API calls happen before persist
|
||||||
|
localStorage.setItem('auth-storage', JSON.stringify(authState));
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Session Security (Redis-Backed)
|
||||||
|
|
||||||
|
**Problem Solved:** User swapping / random logout issues
|
||||||
|
|
||||||
|
**Implementation:**
|
||||||
|
```python
|
||||||
|
# settings.py
|
||||||
|
SESSION_ENGINE = 'django.contrib.sessions.backends.cache'
|
||||||
|
SESSION_CACHE_ALIAS = 'default' # Redis
|
||||||
|
|
||||||
|
# auth/backends.py
|
||||||
|
class NoCacheModelBackend(ModelBackend):
|
||||||
|
"""Authentication backend without user caching"""
|
||||||
|
pass
|
||||||
|
```
|
||||||
|
|
||||||
|
**Session Integrity:**
|
||||||
|
- Stores `account_id` and `user_id` in session
|
||||||
|
- Validates on every request
|
||||||
|
- Prevents cross-request contamination
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## API Key Management
|
||||||
|
|
||||||
|
**Model:** `APIKey` in `auth/models.py`
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| key | CharField | Hashed API key |
|
||||||
|
| account | ForeignKey | Owner account |
|
||||||
|
| site | ForeignKey | Optional: specific site |
|
||||||
|
| name | CharField | Key name/description |
|
||||||
|
| is_active | Boolean | Enable/disable |
|
||||||
|
| created_at | DateTime | Creation time |
|
||||||
|
| last_used_at | DateTime | Last usage time |
|
||||||
|
|
||||||
|
**Generation:**
|
||||||
|
- 32-character random key
|
||||||
|
- Stored hashed (SHA-256)
|
||||||
|
- Shown once on creation
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Permission Checking
|
||||||
|
|
||||||
|
**In ViewSets:**
|
||||||
|
```python
|
||||||
|
class MyViewSet(AccountModelViewSet):
|
||||||
|
permission_classes = [IsAuthenticated]
|
||||||
|
|
||||||
|
def get_queryset(self):
|
||||||
|
# Automatically filtered by request.account
|
||||||
|
return super().get_queryset()
|
||||||
|
```
|
||||||
|
|
||||||
|
**Role Checks:**
|
||||||
|
```python
|
||||||
|
if request.user.is_admin_or_developer:
|
||||||
|
# Admin/developer access
|
||||||
|
pass
|
||||||
|
elif request.user.role == 'editor':
|
||||||
|
# Editor access
|
||||||
|
pass
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Logout Flow
|
||||||
|
|
||||||
|
**Backend:**
|
||||||
|
1. Blacklist refresh token (if using token blacklist)
|
||||||
|
2. Clear session
|
||||||
|
|
||||||
|
**Frontend (Critical):**
|
||||||
|
```typescript
|
||||||
|
logout: () => {
|
||||||
|
// NEVER use localStorage.clear() - breaks Zustand persist
|
||||||
|
const authKeys = ['auth-storage', 'site-storage', 'sector-storage', 'billing-storage'];
|
||||||
|
authKeys.forEach(key => localStorage.removeItem(key));
|
||||||
|
|
||||||
|
// Reset dependent stores
|
||||||
|
useSiteStore.setState({ activeSite: null });
|
||||||
|
useSectorStore.setState({ activeSector: null, sectors: [] });
|
||||||
|
|
||||||
|
set({ user: null, token: null, isAuthenticated: false });
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
| Issue | Cause | Fix |
|
||||||
|
|-------|-------|-----|
|
||||||
|
| 403 after login | Tokens not persisted before API call | Write to localStorage synchronously |
|
||||||
|
| User swapping | DB-backed sessions with user caching | Redis sessions + NoCacheModelBackend |
|
||||||
|
| Token refresh loop | Refresh token expired | Redirect to login |
|
||||||
|
| API key not working | Missing site scope | Check API key has correct site assigned |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
|
||||||
|
| Feature | Status | Description |
|
||||||
|
|---------|--------|-------------|
|
||||||
|
| Token blacklist | 🔜 Planned | Proper refresh token invalidation |
|
||||||
|
| 2FA | 🔜 Planned | Two-factor authentication |
|
||||||
|
| SSO | 🔜 Planned | Google/GitHub OAuth |
|
||||||
@@ -1,73 +0,0 @@
|
|||||||
# Identity and Authentication
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Document how user identity, JWT handling, API keys, and session flows work, including middleware and validation rules.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- JWT utilities: `backend/igny8_core/auth/utils.py`
|
|
||||||
- Account context middleware: `backend/igny8_core/auth/middleware.py`
|
|
||||||
- DRF authentication classes: `backend/igny8_core/api/authentication.py`
|
|
||||||
- DRF settings for auth/throttle: `backend/igny8_core/settings.py`
|
|
||||||
- User model and roles: `backend/igny8_core/auth/models.py`
|
|
||||||
- Auth URLs and views: `backend/igny8_core/auth/urls.py`, `backend/igny8_core/auth/views.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Support multiple auth mechanisms: API key (WordPress bridge), JWT bearer tokens, session auth without CSRF for APIs, and basic auth fallback.
|
|
||||||
- Populate tenant/site context alongside user identity so downstream viewsets enforce isolation.
|
|
||||||
- Enforce active account/plan presence before serving protected endpoints (except admin/auth routes).
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Authentication order (DRF `DEFAULT_AUTHENTICATION_CLASSES` in `settings.py`): API key → JWT → CSRF-exempt session → basic auth. The first class that authenticates sets `request.user`; `request.account` may also be set by API key or JWT.
|
|
||||||
- API Key flow (`APIKeyAuthentication`): expects `Authorization: Bearer <api_key>` that is not JWT-like; finds an active `Site` with `wp_api_key`, loads its `account`, and selects an active user (owner preferred, else any active developer/owner/admin). Sets `request.account` and `request.site`. Rejects short/invalid keys; returns an auth failure if no active user exists.
|
|
||||||
- JWT flow (`JWTAuthentication`): expects `Authorization: Bearer <jwt>`; decodes via `auth.utils.decode_token`; only accepts tokens with `type == access`. Retrieves `User` by `user_id`; optional `account_id` is resolved to `Account` and set on `request.account`. Invalid/expired tokens fall through to other auth classes.
|
|
||||||
- Session flow (`CSRFExemptSessionAuthentication`): uses Django session cookies without CSRF enforcement for API calls.
|
|
||||||
- Basic auth: last resort; does not set tenant context.
|
|
||||||
- Token utilities (`auth/utils.py`) generate and decode access/refresh tokens using expiries from settings (`JWT_ACCESS_TOKEN_EXPIRY`, `JWT_REFRESH_TOKEN_EXPIRY`), embedding `user_id`, `account_id`, `email`, issued/expiry timestamps, and token `type`.
|
|
||||||
- Middleware (`AccountContextMiddleware`) runs on every request except admin/auth paths: refreshes session users from DB to pick up current account/plan, validates presence of an active plan, sets `request.account`, and logs out session users when invalid. For JWT-bearing requests it decodes the token directly and sets `request.account`. If account/plan is missing or inactive, it returns JSON with `success=false` and appropriate HTTP status.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `User` with `role` and `account` FKs in `auth/models.py`.
|
|
||||||
- `Account` with plan and billing fields; plan status is used for access gating.
|
|
||||||
- `Site` with `wp_api_key` for API key auth; `SiteUserAccess` for per-site grants.
|
|
||||||
- `PasswordResetToken` model for password reset flows.
|
|
||||||
- JWT payload fields: `user_id`, `account_id`, `email`, `exp`, `iat`, `type`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Middleware step: `AccountContextMiddleware` determines `request.account` (session or JWT) and validates plan status; skips admin/auth routes.
|
|
||||||
- DRF auth step: API key/JWT/session/basic authenticators run in order, potentially setting `request.account` (API key/JWT) and `request.site` (API key).
|
|
||||||
- Viewsets then apply role/permission checks and tenant/site/sector filtering via base classes in `api/base.py`.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- All module viewsets rely on `request.user` and `request.account` set by the auth stack. Site-aware modules can read `request.site` when API key auth is used.
|
|
||||||
- Role helpers (`is_admin_or_developer`, `is_system_account_user`) influence filtering bypass in base viewsets.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- JWT lifetimes: access tokens default to 15 minutes; refresh tokens to 30 days (configurable in settings).
|
|
||||||
- Session users are refreshed on each request to pick up plan/account changes.
|
|
||||||
- Password reset tokens track expiry and usage via `expires_at` and `used` flags.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Middleware returns JSON errors for missing account or inactive plan and logs out session users in those cases.
|
|
||||||
- Invalid/expired JWTs cause the JWT authenticator to return `None`, allowing other auth methods; decoding errors raise `InvalidTokenError` in utilities.
|
|
||||||
- API key auth raises an auth failure when no active user is available for the resolved account.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- `request.account` is set early; base viewsets enforce account filtering unless user has admin/developer/system-account privileges.
|
|
||||||
- API key auth also sets `request.site` for integration contexts; site/sector filtering occurs in `SiteSectorModelViewSet`.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Active plan is required for access (middleware enforces). Credit debits/charges are handled in billing modules, not in the auth layer.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Token generation/validation is synchronous. Background tasks should receive explicit user/account identifiers in their payloads when invoked.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Authentication stack is ordered to give integration API keys precedence, then JWT for app clients, then session for browser-based flows.
|
|
||||||
- Tenant context must be established before view logic; do not move or remove `AccountContextMiddleware`.
|
|
||||||
- Expiry durations and JWT secrets are centrally configured in `settings.py`.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Use token helpers from `auth/utils.py` when issuing tokens; do not handcraft JWTs.
|
|
||||||
- Mount new auth-sensitive endpoints under existing routers and rely on DRF auth classes instead of custom header parsing.
|
|
||||||
- Ensure new features that require site context can work with API key auth by checking `request.site`.
|
|
||||||
- Keep plan enforcement in place; bypass only for admin/system routes when justified.
|
|
||||||
@@ -1,85 +0,0 @@
|
|||||||
# Data Flow Diagrams (Narrative)
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe end-to-end data movement through the system based on current routing, middleware, and model conventions. No diagrams are embedded; flows are explained textually.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Request routing: `backend/igny8_core/urls.py`
|
|
||||||
- Middleware: `backend/igny8_core/middleware/request_id.py`, `backend/igny8_core/auth/middleware.py`, `backend/igny8_core/middleware/resource_tracker.py`
|
|
||||||
- DRF base viewsets: `backend/igny8_core/api/base.py`
|
|
||||||
- Authentication classes: `backend/igny8_core/api/authentication.py`
|
|
||||||
- Tenancy models: `backend/igny8_core/auth/models.py`
|
|
||||||
- Celery configuration: `backend/igny8_core/settings.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Trace how HTTP requests are processed, tenant-scoped, authorized, and persisted.
|
|
||||||
- Show where async processing departs to Celery and where responses are shaped.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Incoming request → Django middleware stack:
|
|
||||||
- Security/WhiteNoise/CORS/session/common/CSRF/Django auth.
|
|
||||||
- `RequestIDMiddleware` assigns `request.request_id` and returns it in `X-Request-ID`.
|
|
||||||
- `AccountContextMiddleware` resolves user/account (session or JWT) and enforces active plan; sets `request.account`.
|
|
||||||
- `ResourceTrackingMiddleware` optionally tracks resource usage for admin/developer users when the `X-Debug-Resource-Tracking` header is true; adds `X-Resource-Tracking-ID`.
|
|
||||||
- URL dispatch via `urls.py` routes to module routers (auth, account, planner, writer, system, billing, automation, linker, optimizer, publisher, integration) under `/api/v1/*`.
|
|
||||||
- DRF viewset pipeline:
|
|
||||||
- Authentication classes (API key → JWT → session → basic) establish `request.user` (and optionally `request.account`/`request.site`).
|
|
||||||
- Base viewsets (`AccountModelViewSet`, `SiteSectorModelViewSet`) filter querysets by `account`/`site`/`sector` and attach `account` on create.
|
|
||||||
- Serializers handle validation; responses are wrapped by unified helpers to standardize success/error payloads and pagination.
|
|
||||||
- Persistence:
|
|
||||||
- Tenant-scoped models inherit `AccountBaseModel` or `SiteSectorBaseModel`; save hooks enforce account/site/sector alignment.
|
|
||||||
- Soft deletion is used where models implement `soft_delete`, respecting retention windows from account settings.
|
|
||||||
- Async/Background:
|
|
||||||
- Celery uses Redis broker/backend; tasks inherit JSON payloads and time limits from `settings.py`.
|
|
||||||
- Automation, AI, publishing, and billing tasks enqueue via Celery; results return through database/state updates, not synchronous responses.
|
|
||||||
- Response:
|
|
||||||
- Unified response wrappers ensure `success`, `data`/`error`, and request ID are present; paginated responses include `count/next/previous/results`.
|
|
||||||
- Throttling headers apply per-scope (as configured in `REST_FRAMEWORK` throttles).
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Tenancy bases: `AccountBaseModel`, `SiteSectorBaseModel`.
|
|
||||||
- Core entities: `Account`, `Plan`, `Site`, `Sector`, `User`, `SiteUserAccess`.
|
|
||||||
- Module-specific models follow the same tenancy bases (documented in module-specific files).
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
1) HTTP request hits middleware; IDs and tenant context are set.
|
|
||||||
2) DRF authentication authenticates and sets user/account/site.
|
|
||||||
3) Viewset filters data by tenant/site/sector and runs serializer validation.
|
|
||||||
4) DB operations persist data with enforced tenant alignment.
|
|
||||||
5) Optional Celery tasks are queued for long-running work.
|
|
||||||
6) Response returns unified JSON with request IDs and optional throttling/pagination headers.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Auth context set in middleware is consumed by all module viewsets for scoping.
|
|
||||||
- API key auth provides site context for integration/publisher flows.
|
|
||||||
- Celery configuration is shared by automation/AI/publishing/billing task modules.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Entity lifecycle changes (create/update/delete/soft-delete) flow through base viewsets and tenancy bases, ensuring account/site/sector consistency.
|
|
||||||
- Request lifecycle includes request ID creation, optional resource tracking, and unified response wrapping.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Middleware can short-circuit with JSON errors for missing account/plan.
|
|
||||||
- Viewset overrides wrap validation and server errors into unified responses; missing objects return 404 payloads.
|
|
||||||
- Throttling (scope-based) returns standard DRF throttle responses with headers.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All tenant-bound data flows require `request.account`; filtering and save hooks prevent cross-tenant access.
|
|
||||||
- Admin/developer/system-account users may bypass tenant filtering; system accounts are guarded against deletion.
|
|
||||||
- Site/sector alignment is validated on save for models inheriting `SiteSectorBaseModel`.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Plan activation is validated in middleware. Credit debits and billing workflows occur in billing modules (covered elsewhere) after tenant resolution.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Celery broker/backend configuration in `settings.py` governs async flow; tasks should include account/site identifiers to maintain scoping.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Request ID and resource tracking enable traceability and performance debugging.
|
|
||||||
- Middleware ordering ensures tenant context precedes view logic.
|
|
||||||
- Unified response format keeps clients consistent across modules.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Preserve middleware order; new middleware must not break request ID or tenant context.
|
|
||||||
- Ensure new viewsets inherit the base classes to pick up scoping and unified responses.
|
|
||||||
- When adding async tasks, include tenant/site identifiers and respect Celery limits from settings.
|
|
||||||
@@ -1,79 +0,0 @@
|
|||||||
# Multitenancy Model
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Explain how tenant, site, and sector isolation is enforced across models, middleware, and viewsets, based on the current implementation.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Tenant base models: `backend/igny8_core/auth/models.py` (`AccountBaseModel`, `SiteSectorBaseModel`)
|
|
||||||
- Core entities: `backend/igny8_core/auth/models.py` (`Account`, `Plan`, `Site`, `Sector`, `Industry`, `IndustrySector`, `SeedKeyword`, `SiteUserAccess`, `User`)
|
|
||||||
- Middleware for context: `backend/igny8_core/auth/middleware.py`
|
|
||||||
- DRF base viewsets: `backend/igny8_core/api/base.py`
|
|
||||||
- Auth utilities and JWT: `backend/igny8_core/api/authentication.py`, `backend/igny8_core/auth/utils.py`
|
|
||||||
- URL routing (module mounting): `backend/igny8_core/urls.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Enforce per-account isolation for all models carrying an `account` FK.
|
|
||||||
- Enforce per-site and per-sector scoping for content models via `SiteSectorBaseModel`.
|
|
||||||
- Inject tenant context on every request (session or JWT/API key), then apply scoping in base viewsets.
|
|
||||||
- Allow controlled overrides for admins, developers, and system accounts.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- `AccountBaseModel` adds an `account` FK plus timestamps and indexes; all tenant-scoped models inherit this to guarantee account linkage.
|
|
||||||
- `SiteSectorBaseModel` extends `AccountBaseModel` with `site` and `sector` FKs, indexes on `(account, site, sector)`, and a save hook that sets `account` from `site` and validates that `sector` belongs to the same `site`; raises validation errors on mismatch.
|
|
||||||
- `AccountContextMiddleware` sets `request.account` by refreshing the authenticated session user (with account and plan) or by decoding JWT tokens; it rejects requests when account is missing or plan is inactive, returning structured JSON and logging out session users. It skips admin and auth endpoints to avoid interference.
|
|
||||||
- JWT authentication (`api/authentication.py`) decodes tokens and sets `request.account` from the token payload; API key authentication sets both `request.account` and `request.site` for WordPress bridge calls.
|
|
||||||
- `AccountModelViewSet` auto-filters querysets by `account` when models expose that field. It bypasses filtering for admins/developers/system-account users; otherwise, it uses `request.account` or falls back to the authenticated user’s account. Creates set `account` on save when present.
|
|
||||||
- `SiteSectorModelViewSet` extends the above to filter by site/sector if those fields exist, using request query parameters and tenancy context.
|
|
||||||
- `User` role helpers (`is_admin_or_developer`, `is_system_account_user`) and account checks gate override behavior.
|
|
||||||
- `SiteUserAccess` provides explicit per-site access for non-admin roles; `User.get_accessible_sites` respects system account, developer, owner/admin, and granted access rules.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `Account`: tenant container with plan, credits, billing fields, status, and retention settings.
|
|
||||||
- `Plan`: defines limits (max users/sites/industries/author profiles), credit inclusion, Stripe IDs.
|
|
||||||
- `Site`: belongs to an account, optionally an industry; includes status, hosting type, legacy WP fields, and SEO metadata.
|
|
||||||
- `Sector`: belongs to a site and industry sector template; enforces account alignment and plan-based max-sector validation.
|
|
||||||
- `Industry`, `IndustrySector`, `SeedKeyword`: global reference data not bound to a single account.
|
|
||||||
- `SiteUserAccess`: explicit many-to-many grants between users and sites.
|
|
||||||
- `User`: links to account with role-based access flags.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Request enters middleware; `AccountContextMiddleware` determines `request.account` (session or JWT/API key), validating plan status.
|
|
||||||
- Viewsets inheriting from `AccountModelViewSet`/`SiteSectorModelViewSet` filter querysets by `account` (and site/sector when present) before pagination/serialization.
|
|
||||||
- Object creation sets `account` automatically when the serializer’s model has that field; site/sector-based models validate alignment on save.
|
|
||||||
- Admin/developer/system-account users skip account filtering; other users remain constrained.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- All module viewsets depend on the base viewsets for scoping.
|
|
||||||
- Automation, planner, writer, billing, linker, optimizer, publisher, and integration models inherit from the tenancy bases to enforce account/site/sector ownership.
|
|
||||||
- API key flows for WordPress set `request.site`, enabling integration-specific logic to run in a site-aware context.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Account status changes (active, suspended, trial, cancelled) and plan activation directly affect access through middleware plan validation.
|
|
||||||
- Sector creation enforces plan-based limits for active sectors per site.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Middleware returns JSON errors for missing account or inactive plan, with HTTP 403 or 402 semantics and logs out session users.
|
|
||||||
- Base viewsets wrap CRUD operations in unified responses; validation failures or missing objects are returned in structured error payloads.
|
|
||||||
- Save-time validation on `SiteSectorBaseModel` and `Sector` raises validation errors when site/sector alignment or plan limits are violated.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Every tenant-scoped model carries `account`; site/sector-aware models carry `site` and `sector` and must align to the same account.
|
|
||||||
- Middleware populates `request.account`; base viewsets enforce filtering unless the user is an admin/developer/system-account member.
|
|
||||||
- System accounts (`aws-admin`, `default-account`, `default`) and privileged roles can bypass scoping; protected from deletion via guard clauses.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Middleware requires an active plan before allowing requests (except auth/admin paths). Credits, charges, and plan enforcement are handled in billing modules (documented elsewhere).
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Celery tasks inherit tenant context via payloads supplied by calling viewsets/services; the tenancy bases ensure stored records retain `account`/`site`/`sector`.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Tenancy is enforced as early as middleware to avoid leakage in view logic.
|
|
||||||
- Base viewsets centralize scoping and unified responses to reduce duplication across modules.
|
|
||||||
- Role-based overrides exist for ops and system accounts; safeguards prevent system account deletion.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Inherit from `AccountBaseModel` or `SiteSectorBaseModel` for any new tenant/site/sector data models.
|
|
||||||
- Inherit viewsets from `AccountModelViewSet` or `SiteSectorModelViewSet` to get automatic scoping and unified responses.
|
|
||||||
- Do not bypass `AccountContextMiddleware`; ensure new endpoints live under `/api/v1/*` and rely on the auth stack (API key → JWT → session).
|
|
||||||
- Validate that new background tasks carry account/site/sector identifiers so downstream saves remain scoped.
|
|
||||||
@@ -1,61 +0,0 @@
|
|||||||
# Tech Stack
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Document the concrete technologies and dependencies in use across backend and frontend as defined in the repository.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Backend dependency manifest: `backend/requirements.txt`
|
|
||||||
- Backend settings (framework integration): `backend/igny8_core/settings.py`
|
|
||||||
- Frontend dependency manifest: `frontend/package.json`
|
|
||||||
- Frontend build tooling: `frontend/vite.config.ts`, `frontend/tsconfig*.json`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Backend: Django 5.x with DRF for APIs, Celery for async tasks, Redis for broker/result, PostgreSQL or SQLite for data, drf-spectacular for OpenAPI, Stripe/PayPal configs for billing, WhiteNoise for static serving.
|
|
||||||
- Frontend: React 19 with Vite, TypeScript, TailwindCSS, Zustand state, React Router 7, ApexCharts and FullCalendar for UI widgets.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Backend settings wire DRF pagination, filters, authentication (API key, JWT, session, basic), throttling, schema generation, CORS, Celery, logging, and Stripe/PayPal credentials. Static assets are served via WhiteNoise; admin uses Django contrib.
|
|
||||||
- `requirements.txt` enumerates runtime libs: Django, gunicorn, psycopg2-binary (PostgreSQL), redis, WhiteNoise, DRF, django-filter, django-cors-headers, PyJWT, requests, Celery, BeautifulSoup4, psutil, docker (for ops scripts), drf-spectacular, and stripe.
|
|
||||||
- Frontend `package.json` pins React 19, React Router 7, Zustand 5, Vite 6, TailwindCSS 4, ApexCharts 4, FullCalendar 6, react-dnd, dropzone, lucide/react-heroicons, and testing/tooling (Vitest, Testing Library, ESLint).
|
|
||||||
- Build scripts use Vite for dev and production builds, with separate marketing mode; tests via Vitest; lint via ESLint; type-check via `tsc -b`.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Not applicable; this file tracks technology components rather than domain models.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Backend runs under Django/DRF with middleware and installed apps per `settings.py`. ASGI/WSGI entrypoints in `igny8_core/asgi.py` and `igny8_core/wsgi.py` (default WSGI via gunicorn).
|
|
||||||
- Celery worker/beat use Redis URLs from `settings.py` and respect JSON serialization/time limits.
|
|
||||||
- Frontend builds with Vite, consuming environment variables defined via Vite conventions.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- DRF auth classes depend on PyJWT and custom utilities for token handling.
|
|
||||||
- Celery tasks in AI/automation/publishing rely on Redis connectivity and the configured serializer/time limits.
|
|
||||||
- Stripe/PayPal keys in settings are consumed by billing modules.
|
|
||||||
- Frontend API calls rely on the DRF endpoints exposed in `igny8_core/urls.py`.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Dependency-driven: none beyond the build/runtime phases (install → build → serve).
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Backend error handling configured via custom DRF exception handler (enabled by default) and logging setup in `settings.py`.
|
|
||||||
- Frontend build/test errors are surfaced through Vite/TypeScript/Vitest/ESLint tooling.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Implemented at runtime by backend middleware and viewsets; the tech stack provides JWT, session, and API key support to carry tenant context.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Stripe and PayPal keys in `settings.py` enable billing integrations; credit logic is implemented in billing modules (documented elsewhere).
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Celery configured in `settings.py` with Redis broker/backend, JSON serialization, and task limits; beat scheduling persists in `celerybeat-schedule`.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Keep dependency manifests authoritative (`requirements.txt`, `package.json`).
|
|
||||||
- Redis is the default async backbone; Postgres is the default DB with SQLite fallback for local/dev.
|
|
||||||
- Vite + React 19 selected for fast dev/build; TailwindCSS 4 used for styling; Zustand for state.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Add backend dependencies to `requirements.txt` and pin versions appropriately; update settings if new middleware/auth is added.
|
|
||||||
- Add frontend dependencies to `package.json`; run `npm install` and ensure Vite/TSC builds remain clean.
|
|
||||||
- Respect configured auth stack (API key → JWT → session) when adding API clients.
|
|
||||||
- Keep CORS and env vars aligned with the domains/ports in use for local and production.
|
|
||||||
299
docs/00-SYSTEM/TENANCY.md
Normal file
299
docs/00-SYSTEM/TENANCY.md
Normal file
@@ -0,0 +1,299 @@
|
|||||||
|
# Multi-Tenancy Architecture
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Backend Path:** `backend/igny8_core/auth/models.py`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Data Hierarchy
|
||||||
|
|
||||||
|
```
|
||||||
|
Account (Tenant)
|
||||||
|
├── Users (team members)
|
||||||
|
├── Subscription (plan binding)
|
||||||
|
├── Sites
|
||||||
|
│ ├── Sectors
|
||||||
|
│ │ ├── Keywords
|
||||||
|
│ │ ├── Clusters
|
||||||
|
│ │ ├── ContentIdeas
|
||||||
|
│ │ ├── Tasks
|
||||||
|
│ │ ├── Content
|
||||||
|
│ │ └── Images
|
||||||
|
│ └── Integrations (WordPress)
|
||||||
|
└── Billing (credits, transactions)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Core Models
|
||||||
|
|
||||||
|
### Account
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| name | CharField | Account/organization name |
|
||||||
|
| is_active | Boolean | Enable/disable account |
|
||||||
|
| credits | Decimal | Current credit balance |
|
||||||
|
| stripe_customer_id | CharField | Stripe integration |
|
||||||
|
| paypal_customer_id | CharField | PayPal integration |
|
||||||
|
| created_at | DateTime | Registration date |
|
||||||
|
|
||||||
|
### Plan
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| name | CharField | Plan name (Free, Starter, Growth, Scale) |
|
||||||
|
| included_credits | Integer | Monthly credit allocation |
|
||||||
|
| max_sites | Integer | Site limit |
|
||||||
|
| max_users | Integer | User limit |
|
||||||
|
| max_keywords | Integer | Keyword limit |
|
||||||
|
| max_clusters | Integer | Cluster limit |
|
||||||
|
| max_content_ideas | Integer | Monthly idea limit |
|
||||||
|
| max_content_words | Integer | Monthly word limit |
|
||||||
|
| max_images_basic | Integer | Monthly basic image limit |
|
||||||
|
| max_images_premium | Integer | Monthly premium image limit |
|
||||||
|
| is_active | Boolean | Available for purchase |
|
||||||
|
| is_internal | Boolean | Internal/test plan |
|
||||||
|
|
||||||
|
### Site
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | ForeignKey | Owner account |
|
||||||
|
| name | CharField | Site name |
|
||||||
|
| domain | CharField | Primary domain |
|
||||||
|
| is_active | Boolean | Enable/disable |
|
||||||
|
|
||||||
|
### Sector
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| site | ForeignKey | Parent site |
|
||||||
|
| account | ForeignKey | Owner account |
|
||||||
|
| industry | ForeignKey | Industry template |
|
||||||
|
| name | CharField | Sector name |
|
||||||
|
| is_active | Boolean | Enable/disable |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Base Model Classes
|
||||||
|
|
||||||
|
### AccountBaseModel
|
||||||
|
|
||||||
|
**File:** `auth/models.py`
|
||||||
|
|
||||||
|
All account-scoped models inherit from this:
|
||||||
|
|
||||||
|
```python
|
||||||
|
class AccountBaseModel(models.Model):
|
||||||
|
account = models.ForeignKey(Account, on_delete=models.CASCADE)
|
||||||
|
|
||||||
|
class Meta:
|
||||||
|
abstract = True
|
||||||
|
```
|
||||||
|
|
||||||
|
**Used by:** Billing records, Settings, API keys
|
||||||
|
|
||||||
|
### SiteSectorBaseModel
|
||||||
|
|
||||||
|
**File:** `auth/models.py`
|
||||||
|
|
||||||
|
All content models inherit from this:
|
||||||
|
|
||||||
|
```python
|
||||||
|
class SiteSectorBaseModel(AccountBaseModel):
|
||||||
|
site = models.ForeignKey(Site, on_delete=models.CASCADE)
|
||||||
|
sector = models.ForeignKey(Sector, on_delete=models.CASCADE)
|
||||||
|
|
||||||
|
class Meta:
|
||||||
|
abstract = True
|
||||||
|
```
|
||||||
|
|
||||||
|
**Used by:** Keywords, Clusters, Ideas, Tasks, Content, Images
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ViewSet Base Classes
|
||||||
|
|
||||||
|
### AccountModelViewSet
|
||||||
|
|
||||||
|
**File:** `api/base.py`
|
||||||
|
|
||||||
|
Automatically filters queryset by account:
|
||||||
|
|
||||||
|
```python
|
||||||
|
class AccountModelViewSet(ModelViewSet):
|
||||||
|
def get_queryset(self):
|
||||||
|
qs = super().get_queryset()
|
||||||
|
if not self.request.user.is_admin_or_developer:
|
||||||
|
qs = qs.filter(account=self.request.account)
|
||||||
|
return qs
|
||||||
|
```
|
||||||
|
|
||||||
|
### SiteSectorModelViewSet
|
||||||
|
|
||||||
|
**File:** `api/base.py`
|
||||||
|
|
||||||
|
Filters by account + site + sector:
|
||||||
|
|
||||||
|
```python
|
||||||
|
class SiteSectorModelViewSet(AccountModelViewSet):
|
||||||
|
def get_queryset(self):
|
||||||
|
qs = super().get_queryset()
|
||||||
|
site_id = self.request.query_params.get('site_id')
|
||||||
|
sector_id = self.request.query_params.get('sector_id')
|
||||||
|
if site_id:
|
||||||
|
qs = qs.filter(site_id=site_id)
|
||||||
|
if sector_id:
|
||||||
|
qs = qs.filter(sector_id=sector_id)
|
||||||
|
return qs
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Where Tenancy Applies
|
||||||
|
|
||||||
|
### Uses Site/Sector Filtering
|
||||||
|
|
||||||
|
| Module | Filter |
|
||||||
|
|--------|--------|
|
||||||
|
| Planner (Keywords, Clusters, Ideas) | site + sector |
|
||||||
|
| Writer (Tasks, Content, Images) | site + sector |
|
||||||
|
| Linker | site + sector |
|
||||||
|
| Optimizer | site + sector |
|
||||||
|
| Setup/Add Keywords | site + sector |
|
||||||
|
|
||||||
|
### Account-Level Only (No Site/Sector)
|
||||||
|
|
||||||
|
| Module | Filter |
|
||||||
|
|--------|--------|
|
||||||
|
| Billing/Plans | account only |
|
||||||
|
| Account Settings | account only |
|
||||||
|
| Team Management | account only |
|
||||||
|
| User Profile | user only |
|
||||||
|
| System Settings | account only |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Frontend Implementation
|
||||||
|
|
||||||
|
### Site Selection
|
||||||
|
|
||||||
|
**Store:** `store/siteStore.ts`
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const useSiteStore = create({
|
||||||
|
activeSite: Site | null,
|
||||||
|
sites: Site[],
|
||||||
|
loadSites: () => Promise<void>,
|
||||||
|
setActiveSite: (site: Site) => void,
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### Sector Selection
|
||||||
|
|
||||||
|
**Store:** `store/sectorStore.ts`
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const useSectorStore = create({
|
||||||
|
activeSector: Sector | null,
|
||||||
|
sectors: Sector[],
|
||||||
|
loadSectorsForSite: (siteId: number) => Promise<void>,
|
||||||
|
setActiveSector: (sector: Sector) => void,
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### Sector Loading Pattern
|
||||||
|
|
||||||
|
Sectors are loaded by `PageHeader` component, not `AppLayout`:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// PageHeader.tsx
|
||||||
|
useEffect(() => {
|
||||||
|
if (hideSiteSector) return; // Skip for account pages
|
||||||
|
|
||||||
|
if (activeSite?.id && activeSite?.is_active) {
|
||||||
|
loadSectorsForSite(activeSite.id);
|
||||||
|
}
|
||||||
|
}, [activeSite?.id, hideSiteSector]);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Pages with `hideSiteSector={true}`:**
|
||||||
|
- `/account/*` (settings, team, billing)
|
||||||
|
- User profile pages
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Global Resources (No Tenancy)
|
||||||
|
|
||||||
|
These are system-wide, not tenant-specific:
|
||||||
|
|
||||||
|
| Model | Purpose |
|
||||||
|
|-------|---------|
|
||||||
|
| Industry | Global industry taxonomy |
|
||||||
|
| IndustrySector | Sub-categories within industries |
|
||||||
|
| SeedKeyword | Global keyword database |
|
||||||
|
| GlobalIntegrationSettings | Platform API keys |
|
||||||
|
| GlobalAIPrompt | Default prompt templates |
|
||||||
|
| GlobalAuthorProfile | Author persona templates |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Tenant Data vs System Data
|
||||||
|
|
||||||
|
### System Data (KEEP on reset)
|
||||||
|
|
||||||
|
- Plans, CreditPackages
|
||||||
|
- Industries, IndustrySectors
|
||||||
|
- GlobalIntegrationSettings
|
||||||
|
- GlobalAIPrompt, GlobalAuthorProfile
|
||||||
|
- CreditCostConfig
|
||||||
|
- PaymentMethodConfig
|
||||||
|
|
||||||
|
### Tenant Data (CLEAN on reset)
|
||||||
|
|
||||||
|
- Accounts, Users, Sites, Sectors
|
||||||
|
- Keywords, Clusters, Ideas, Tasks, Content, Images
|
||||||
|
- Subscriptions, Invoices, Payments
|
||||||
|
- CreditTransactions, CreditUsageLogs
|
||||||
|
- SiteIntegrations, SyncEvents
|
||||||
|
- AutomationConfigs, AutomationRuns
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Admin/Developer Bypass
|
||||||
|
|
||||||
|
Admin and developer users can access all tenants:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# In auth/models.py
|
||||||
|
class User:
|
||||||
|
@property
|
||||||
|
def is_admin_or_developer(self):
|
||||||
|
return self.role in ['admin', 'developer']
|
||||||
|
```
|
||||||
|
|
||||||
|
**Bypass Rules:**
|
||||||
|
- Admin: Full access to own account
|
||||||
|
- Developer: Full access to ALL accounts
|
||||||
|
- System accounts protected from deletion
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
| Issue | Cause | Fix |
|
||||||
|
|-------|-------|-----|
|
||||||
|
| Data leak between accounts | Missing account filter | Use AccountModelViewSet |
|
||||||
|
| Wrong site data | Site not validated for account | Validate site.account == request.account |
|
||||||
|
| Sector not loading | Site changed but sector not reset | Clear activeSector when activeSite changes |
|
||||||
|
| Admin can't see all data | Incorrect role check | Check is_admin_or_developer |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
|
||||||
|
| Feature | Status | Description |
|
||||||
|
|---------|--------|-------------|
|
||||||
|
| Account switching | 🔜 Planned | Allow users to switch between accounts |
|
||||||
|
| Sub-accounts | 🔜 Planned | Hierarchical account structure |
|
||||||
@@ -1,110 +0,0 @@
|
|||||||
# Domain Models
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe the key backend models, their responsibilities, constraints, and tenancy rules, grounded in current implementations.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Tenancy bases and identity: `backend/igny8_core/auth/models.py`
|
|
||||||
- Planner models: `backend/igny8_core/business/planning/models.py`
|
|
||||||
- Writer/content models: `backend/igny8_core/business/content/models.py`
|
|
||||||
- Automation models: `backend/igny8_core/business/automation/models.py`
|
|
||||||
- Billing models: `backend/igny8_core/business/billing/models.py`
|
|
||||||
- Integration models: `backend/igny8_core/business/integration/models.py`
|
|
||||||
- Publishing models: `backend/igny8_core/business/publishing/models.py`
|
|
||||||
- Optimization models: `backend/igny8_core/business/optimization/models.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Provide tenant-scoped storage for planner (keywords/clusters/ideas), writer (tasks/content/taxonomies/images/attributes), automation runs/config, billing (credits, invoices, payments), integration metadata, publishing records, and optimization tasks.
|
|
||||||
- Enforce account/site/sector alignment via base classes and save-time validation.
|
|
||||||
- Track external platform links (WordPress/Shopify/custom), credit usage, and publishing/optimization state.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
### Tenancy Bases (auth/models.py)
|
|
||||||
- `AccountBaseModel`: adds `account`, timestamps, and indexes; all tenant models inherit.
|
|
||||||
- `SiteSectorBaseModel`: extends with `site` and `sector`; save enforces site → account alignment and sector belonging to the same site; raises validation errors if mismatched.
|
|
||||||
|
|
||||||
### Planner (business/planning/models.py)
|
|
||||||
- `Clusters`: tenant/site/sector-scoped keyword group; tracks counts, volume, mapped pages, status (`new/mapped`), disable flag; unique per site/sector by name; soft-deletable.
|
|
||||||
- `Keywords`: tenant/site/sector-scoped keyword tied to a global `SeedKeyword`; optional overrides for volume/difficulty/attributes; optional cluster link (same sector enforced); validation ensures seed keyword industry/sector matches site/sector; status (`new/mapped`), disable flag; soft-deletable.
|
|
||||||
- `ContentIdeas`: ideas tied to clusters and optional keywords; tracks status (`new/queued/completed`), content type/structure, estimated word count; soft-deletable.
|
|
||||||
|
|
||||||
### Writer / Content (business/content/models.py)
|
|
||||||
- `Tasks`: queue items for content generation; tied to cluster (required) and optional idea/taxonomy; content type/structure, keywords text, target word count, status (`queued/completed`); soft-deletable.
|
|
||||||
- `Content`: generated or imported content; stores HTML, word count, SEO fields, cluster link, content type/structure, taxonomy M2M, external IDs/URLs/metadata, sync status, source (`igny8/wordpress`), and status (`draft/review/published`); soft-deletable.
|
|
||||||
- `ContentTaxonomy`: simplified taxonomy (category/tag) with external taxonomy/ID, sync status, description, count, metadata; unique per site by slug/type and by external ID/taxonomy.
|
|
||||||
- `Images`: images linked to content or task; auto-populates account/site/sector from the linked object; tracks type, URL/path, prompt, status, position; soft-deletable.
|
|
||||||
- `ContentClusterMap`: maps content/tasks to clusters with role (`hub/supporting/attribute`) and source (`blueprint/manual/import`); auto-populates tenant context from linked content/task; unique per content+cluster+role.
|
|
||||||
- `ContentAttribute` (alias `ContentAttributeMap`): tenant/site/sector-scoped attributes for content/task/cluster; typed (`product_spec/service_modifier/semantic_facet`), with optional external IDs, sources, and metadata; auto-populates tenant context from linked content/task.
|
|
||||||
|
|
||||||
### Automation (business/automation/models.py)
|
|
||||||
- `AutomationConfig`: per-site config with enable flag, frequency (`daily/weekly/monthly`), scheduled time, batch sizes per stage, within/between-stage delays, and next/last run timestamps.
|
|
||||||
- `AutomationRun`: tracks each run with trigger (`manual/scheduled`), status (`running/paused/cancelled/completed/failed`), current stage, pause/cancel timestamps, start/end, total credits used, per-stage JSON results, and optional error message.
|
|
||||||
|
|
||||||
### Billing (business/billing/models.py)
|
|
||||||
- `CreditTransaction`: ledger of credit changes (purchase/subscription/refund/deduction/adjustment) with balance-after and metadata.
|
|
||||||
- `CreditUsageLog`: detailed AI usage log with operation type (clustering/idea/content/image/reparse/legacy names), credits used, optional cost/model/tokens, related object references, and metadata.
|
|
||||||
- `CreditCostConfig`: admin-configurable credit costs per operation with unit (per request/words/items/images), display metadata, active flag, audit fields, and previous cost tracking.
|
|
||||||
- `Invoice`: tenant invoice with amounts, status (`draft/pending/paid/void/uncollectible`), dates, subscription link, line items JSON, payment metadata, Stripe IDs, notes; helper properties mirror legacy fields.
|
|
||||||
- `Payment`: payment records per invoice with status lifecycle (pending/processing/succeeded/completed/failed/refunded/cancelled), method (Stripe/PayPal/bank/local wallet/manual), provider references, manual notes/approval fields, failure reason, timestamps, metadata.
|
|
||||||
- `CreditPackage`: one-time credit bundles with price, discount, Stripe/PayPal IDs, active/featured flags, description/features, sort order.
|
|
||||||
- `PaymentMethodConfig`: per-country payment-method availability and display/instruction fields; includes bank/local wallet metadata; unique per country+method.
|
|
||||||
- `AccountPaymentMethod`: account-level payment metadata (non-sensitive) with type, display name, default/enabled/verified flags, country code, instructions, metadata; unique per account+display name.
|
|
||||||
|
|
||||||
### Integration (business/integration/models.py)
|
|
||||||
- `SiteIntegration`: tenant/site-specific integration config with platform (`wordpress/shopify/custom`), platform type (`cms/ecommerce/custom_api`), config JSON, credentials JSON, active/sync flags, sync status, last sync/error, timestamps; unique per site+platform.
|
|
||||||
- `SyncEvent`: event log per integration/site with event/action types, success flag, optional content/external IDs, details JSON, error, duration, and timestamps; indexed for debugging feeds.
|
|
||||||
|
|
||||||
### Publishing (business/publishing/models.py)
|
|
||||||
- `PublishingRecord`: tracks content publishing to destinations (wordpress/sites/shopify) with destination IDs/URLs, status (`pending/publishing/published/failed`), timestamps, errors, metadata; site/sector scoped via base.
|
|
||||||
- `DeploymentRecord`: tracks site deployments (sites renderer) with version/deployed_version, status (`pending/deploying/deployed/failed/rolled_back`), deployment URL, error, metadata, timestamps; site/sector scoped.
|
|
||||||
|
|
||||||
### Optimization (business/optimization/models.py)
|
|
||||||
- `OptimizationTask`: content optimization runs with before/after scores and HTML, status (`pending/running/completed/failed`), credits used, metadata; auto-sets account from content; tenant scoped.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Tenant context is inherited from base models; many save methods propagate account/site/sector from related entities (e.g., Images, ContentClusterMap, ContentAttribute).
|
|
||||||
- Planner → Writer linkage: Keywords and Clusters feed ContentIdeas; Tasks reference clusters/ideas; Content references clusters and taxonomies; Images/Attributes link to Tasks/Content.
|
|
||||||
- Automation runs reference planner/writer models and record per-stage outputs; configs control batching/delays.
|
|
||||||
- Billing logs and cost configs govern credit debits triggered by services (see services doc).
|
|
||||||
- Integration/publishing models bind site integrations and publishing deployments to site-scoped content.
|
|
||||||
- Optimization tasks attach to content and capture before/after artifacts.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Planner and writer share clusters/ideas/tasks/content relationships.
|
|
||||||
- Billing models are invoked by services during AI/automation/image/content operations.
|
|
||||||
- Integration events reference content IDs and external IDs for sync traces.
|
|
||||||
- Publishing records reference writer content; deployment records reference sites.
|
|
||||||
- Optimization tasks reference writer content and can influence publishing readiness downstream.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Soft-delete is available on planner keywords/clusters/ideas and writer tasks/content/images via `SoftDeletableModel`.
|
|
||||||
- Status fields track lifecycle: planner (`new/mapped/queued/completed`), writer tasks (`queued/completed`), content (`draft/review/published`), automation (`running/paused/cancelled/completed/failed`), publishing/deployment statuses, payment/invoice statuses, optimization statuses.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Save-time validation in `SiteSectorBaseModel` and `Keywords` ensures tenant/site/sector alignment and industry/sector matching.
|
|
||||||
- Unique constraints prevent duplicate clusters/keywords per site/sector and overlapping taxonomies/external IDs.
|
|
||||||
- Automation runs store error messages and partial stage results; publishing/deployment records store error text.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All models shown are tenant scoped via `AccountBaseModel` or `SiteSectorBaseModel`; save hooks propagate context from related objects where needed.
|
|
||||||
- Privileged roles can bypass filtering at the viewset layer, but persisted records retain account/site/sector ownership.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Credits reside on `Account`; transactions/usage logs record debits/credits; cost configs define per-operation pricing.
|
|
||||||
- Invoices/payments/credit packages configure monetary flows; payment methods can be toggled per country or per account.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Automation configs drive scheduled runs; automation runs record stage outputs and timing.
|
|
||||||
- Publishing/optimization tasks may be executed async via services/Celery (see services doc).
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Tenant isolation is encoded at the model layer via base classes and validation, ensuring downstream services inherit scoping.
|
|
||||||
- Cross-module links (clusters ↔ tasks ↔ content ↔ publishing/optimization) keep content lifecycle traceable.
|
|
||||||
- Billing and integration models include metadata fields to avoid schema churn while capturing provider-specific details.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Inherit new tenant models from `AccountBaseModel` or `SiteSectorBaseModel` to enforce scoping automatically.
|
|
||||||
- Validate cross-entity alignment (site/sector/industry) when relating planner and writer records.
|
|
||||||
- Use existing status fields/choices when extending lifecycles; preserve unique constraints when adding fields.
|
|
||||||
- When integrating new providers, extend or add models parallel to `SiteIntegration`/`SyncEvent` and keep platform-specific data in JSON fields.
|
|
||||||
|
|
||||||
@@ -1,82 +0,0 @@
|
|||||||
# Backend Architecture
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Explain how the backend is structured, wired, and executed across Django/DRF, middleware, routing, async processing, and logging.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Settings and app wiring: `backend/igny8_core/settings.py`
|
|
||||||
- URL routing: `backend/igny8_core/urls.py`
|
|
||||||
- Middleware: `backend/igny8_core/middleware/request_id.py`, `backend/igny8_core/auth/middleware.py`, `backend/igny8_core/middleware/resource_tracker.py`
|
|
||||||
- Auth stack: `backend/igny8_core/api/authentication.py`, `backend/igny8_core/auth/utils.py`
|
|
||||||
- Base viewsets and unified responses: `backend/igny8_core/api/base.py`
|
|
||||||
- Domain models: `backend/igny8_core/auth/models.py` plus `backend/igny8_core/business/*/models.py`
|
|
||||||
- Async/Celery config: `backend/igny8_core/settings.py`
|
|
||||||
- Logging: `backend/igny8_core/settings.py` (publish/webhook logs), automation logging (`backend/igny8_core/business/automation/services/automation_logger.py`)
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Django/DRF API surface under `/api/v1/*` for planner, writer, system, billing, automation, linker, optimizer, publisher, and integration modules.
|
|
||||||
- Middleware establishes request IDs, tenant context, and optional resource tracking before views run.
|
|
||||||
- DRF configuration standardizes pagination, filtering, authentication, throttling, and exception handling.
|
|
||||||
- Celery + Redis provide async execution for AI, automation, publishing, and other long-running tasks.
|
|
||||||
- Logging and CORS/security settings are centralized in settings.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Apps registered in `INSTALLED_APPS` include auth, AI framework, planner, writer, system, billing, automation, optimization, publishing, integration, linker, optimizer, publisher. Custom admin config is loaded first.
|
|
||||||
- Middleware order: security → WhiteNoise → CORS → session/common/CSRF → Django auth → `RequestIDMiddleware` → `AccountContextMiddleware` (tenant + plan enforcement) → `ResourceTrackingMiddleware` (opt-in for admin/developer with header) → messages → clickjacking.
|
|
||||||
- Routing (`urls.py`): admin plus CSV helpers for industries/seed keywords, then `/api/v1/` routers for auth, account, planner, writer, system, billing (user/admin), automation, linker, optimizer, publisher, integration. OpenAPI served at `/api/docs` and `/api/redoc`.
|
|
||||||
- DRF defaults: unified exception handler (unless env disables), custom pagination, filtering/search/ordering, auth stack (API key → JWT → CSRF-exempt session → basic), scoped throttling per operation class, drf-spectacular schema generation with tag ordering.
|
|
||||||
- CORS: IGNY8 domains and local dev ports allowed; credentials enabled; request/resource tracking headers exposed.
|
|
||||||
- Celery: Redis broker/result; JSON serialization; task/soft time limits; single-prefetch; sentinel/SSL toggles via env.
|
|
||||||
- Logging: console plus rotating files for publish/sync, WordPress API, webhooks; request IDs from middleware surface in responses; automation has dedicated file-based logger per run.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Tenancy/identity: `Account`, `Plan`, `Subscription`, `Site`, `Sector`, `User`, `SiteUserAccess`, `PasswordResetToken` (in auth models).
|
|
||||||
- Domain models across planner, writer, billing, automation, publishing, integration, optimization (see domain models doc).
|
|
||||||
- Middleware uses request-scoped `request.account`, `request.site`, and `request.request_id`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
1) Request enters middleware stack; IDs and tenant context are set; plan is verified.
|
|
||||||
2) DRF auth classes authenticate user/api-key/JWT/session.
|
|
||||||
3) Base viewsets filter by tenant/site/sector and handle unified responses.
|
|
||||||
4) Serializers validate; database writes enforce tenant alignment via model bases.
|
|
||||||
5) Optional Celery tasks are enqueued for long-running work; responses remain synchronous JSON with pagination/throttle headers when applicable.
|
|
||||||
6) Logging writes to configured handlers; request/resource IDs are added to responses.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Tenant context from middleware is consumed by all module viewsets.
|
|
||||||
- AI and automation invoke Celery tasks and AI functions; billing services deduct credits used by automation/AI flows.
|
|
||||||
- Integration/publishing modules rely on API key auth to set `request.site` for WordPress and other platforms.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Account/plan validity is checked per request; system accounts are protected from deletion.
|
|
||||||
- Soft-delete is available on many models via `SoftDeletableModel`.
|
|
||||||
- Request lifecycle includes request ID creation, optional resource tracking, and unified response formatting.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Custom DRF exception handler wraps errors with `success=false`.
|
|
||||||
- `AccountContextMiddleware` blocks requests lacking account or active plan (403/402 JSON).
|
|
||||||
- Base viewsets wrap validation/404/500 errors into unified payloads.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Request-level `account` (and `site`/`sector` where applicable) is injected by middleware/auth; base viewsets enforce filtering.
|
|
||||||
- Admin/developer/system-account users bypass tenant filtering; system accounts are guarded from deletion.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Billing env keys configured in settings; plan enforcement occurs in middleware; credit debits handled by billing services during operations.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Celery worker/beat use Redis URLs from settings; task limits and JSON serialization enforced.
|
|
||||||
- Automation, AI, publishing, and optimization tasks run async; automation logger writes per-run files.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Middleware-first tenant enforcement ensures isolation.
|
|
||||||
- Unified API standards (responses, throttles, schema) keep clients consistent.
|
|
||||||
- Celery offloads expensive AI/automation/publishing work with bounded execution.
|
|
||||||
- Logging and request IDs aid observability; resource tracking is opt-in for admins.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Register new apps in `INSTALLED_APPS` and route them under `/api/v1/*`.
|
|
||||||
- Use existing middleware order; add new middleware only if it does not break account/request ID handling.
|
|
||||||
- Inherit from base viewsets for scoping and response consistency.
|
|
||||||
- Use Celery for long-running tasks; respect Redis/task time-limit settings.
|
|
||||||
- Keep env vars (DB, JWT, Redis, Stripe/PayPal) set per `settings.py`; avoid hardcoded secrets.
|
|
||||||
@@ -1,96 +0,0 @@
|
|||||||
# Services and Modules
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe the backend service layer and module wiring that orchestrate domain models, AI/automation, billing, integration, and publishing.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Automation services: `backend/igny8_core/business/automation/services/automation_service.py`, `automation_logger.py`
|
|
||||||
- Billing services: `backend/igny8_core/business/billing/services/credit_service.py`, `invoice_service.py`, `payment_service.py`
|
|
||||||
- Integration service: `backend/igny8_core/business/integration/services/integration_service.py`
|
|
||||||
- Additional service directories (see module-specific docs for details):
|
|
||||||
- Planner: `backend/igny8_core/business/planning/services/*`
|
|
||||||
- Content/Writer: `backend/igny8_core/business/content/services/*`
|
|
||||||
- Automation tasks: `backend/igny8_core/business/automation/tasks.py`
|
|
||||||
- Integration sync: `backend/igny8_core/business/integration/services/*`
|
|
||||||
- Publishing: `backend/igny8_core/business/publishing/services/*`
|
|
||||||
- Optimization: `backend/igny8_core/business/optimization/services/*`
|
|
||||||
- Linking: `backend/igny8_core/business/linking/services/*`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Orchestrate multi-stage automation that chains planner and writer operations using AI and credits.
|
|
||||||
- Manage credit pricing, balance checks, deductions, and ledger logging.
|
|
||||||
- Generate invoices and handle billing documents.
|
|
||||||
- Create, update, test, and list site integrations for external platforms.
|
|
||||||
- Provide domain-specific service hooks for planner, writer, publishing, linking, and optimization flows (behavior documented in module-specific files).
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
### Automation
|
|
||||||
- `AutomationService` enforces single concurrent run per site via cache lock, estimates required credits, checks account balance (with buffer), and creates an `AutomationRun`. It sequences stages (keywords→clusters, clusters→ideas, ideas→tasks/content, image prompt generation, image generation queue) using AI functions (`AutoClusterFunction`, `GenerateIdeasFunction`, `GenerateContentFunction`, `GenerateImagePromptsFunction`) through `AIEngine`, and the Celery image queue (`process_image_generation_queue`). It supports pause/cancel checks mid-stage, records partial progress, and advances `current_stage` with per-stage result JSON and credit tallies. Batch sizes and delays respect `AutomationConfig`.
|
|
||||||
- `AutomationLogger` creates per-run directories (and optional shared mirrors), generates run IDs, writes main/stage logs, mirrors to shared folders when configured, and emits structured JSONL trace events for run start, progress, completion, and errors.
|
|
||||||
|
|
||||||
### Billing
|
|
||||||
- `CreditService` computes credit cost by first consulting `CreditCostConfig` (unit-aware for words/items/images) and falling back to constants. It checks balances, deducts credits atomically, updates account balance, writes `CreditTransaction`, and logs usage in `CreditUsageLog` with optional model/token metadata. It can also add credits and provides legacy check helpers.
|
|
||||||
- `InvoiceService` generates invoice numbers per account/month, creates invoices for subscriptions or credit packages (adding line items and computing totals), supports custom invoices, and marks invoices paid or void. PDF generation is currently a placeholder.
|
|
||||||
- `PaymentService` (see file) processes payments against invoices and updates statuses; details are documented in the module file.
|
|
||||||
|
|
||||||
### Integration
|
|
||||||
- `IntegrationService` creates, updates, deletes, fetches, and lists `SiteIntegration` records, setting account/site automatically. It can test connections per platform (WordPress, Shopify) and delegates to platform-specific test helpers inside the service; unimplemented platforms raise `NotImplementedError`. Credentials are set via `set_credentials`, which currently stores JSON as-is.
|
|
||||||
- Additional sync services (`content_sync_service.py`, `sync_service.py`, `sync_metadata_service.py`, `sync_health_service.py`) coordinate publish/sync flows and health checks; see module docs for specifics.
|
|
||||||
|
|
||||||
### Other Service Areas (structure)
|
|
||||||
- Planner services (`clustering_service.py`, `ideas_service.py`) handle clustering/idea logic.
|
|
||||||
- Content services (`content_generation_service.py`, `content_pipeline_service.py`, `metadata_mapping_service.py`, `validation_service.py`) manage content generation, pipelines, metadata mapping, and validation.
|
|
||||||
- Publishing services (`publisher_service.py`, `deployment_service.py`, `adapters/`) manage publishing/deployment flows and destination adapters.
|
|
||||||
- Optimization services (`analyzer.py`, `optimizer_service.py`) analyze and optimize content.
|
|
||||||
- Linking services (`candidate_engine.py`, `injection_engine.py`, `linker_service.py`) prepare and apply link suggestions.
|
|
||||||
- Automation tasks (`business/automation/tasks.py`) provide Celery entrypoints for automation runs.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Automation: `AutomationService.start_automation` acquires lock → credit estimate/check → create `AutomationRun` → stage methods query planner/writer models, call AI functions via `AIEngine`, respect batch sizes/delays, and update run state/logs → credits are tallied from `AITaskLog` differences.
|
|
||||||
- Billing: operations call `CreditService.check_*`/`deduct_*` before AI or content operations; invoices are created through `InvoiceService` and payments processed via `PaymentService`.
|
|
||||||
- Integration: API endpoints invoke `IntegrationService` to persist integrations, retrieve lists, and run connection tests; sync services handle subsequent data movement.
|
|
||||||
- Other domains: planner/content/publishing/linking/optimization services orchestrate their models and, where applicable, AI or external adapters; see domain docs for invocation points.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Automation stages consume planner (Keywords/Clusters/ContentIdeas) and writer (Tasks/Content/Images) data and rely on credit usage logs from AI tasks.
|
|
||||||
- Billing services are used by automation/AI flows to enforce credit availability and record deductions.
|
|
||||||
- Integration services connect site data and publishing/sync flows to external platforms; publishing services depend on integration metadata when targeting destinations.
|
|
||||||
- Planner/content services feed data used by publishing and optimization tasks.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Automation runs move through stages, can pause/cancel, and record partial progress with stage result JSON.
|
|
||||||
- Credit balances mutate through add/deduct operations; transactions/usage logs capture each change.
|
|
||||||
- Invoices progress through draft/pending/paid/void and payments through their status lifecycle.
|
|
||||||
- Integrations toggle active/sync flags and update sync status/errors.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Automation: checks for concurrent runs; validates minimum keywords; pauses/cancels mid-stage; writes stage error messages; releases locks on failures.
|
|
||||||
- Billing: raises when credit cost unknown or balance insufficient; wraps changes in atomic transactions.
|
|
||||||
- Integration: platform test errors are logged and returned with `success=false`; unsupported platforms raise `NotImplementedError`.
|
|
||||||
- Invoice service prevents voiding paid invoices and returns placeholder PDF until implemented.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Services operate on tenant-scoped models; constructors typically receive account/site or derive them from models. Integration creation sets account from site; credit operations mutate `Account.credits`.
|
|
||||||
- Privileged role bypass applies at the viewset layer; persisted records maintain account/site ownership.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Costs resolved via `CreditCostConfig` (preferred) or constants; units can be per request, words (100/200), item, or image.
|
|
||||||
- Deduct operations both adjust `Account.credits` and log to `CreditTransaction` and `CreditUsageLog`.
|
|
||||||
- Invoice creation links to subscriptions or credit packages and uses account billing email/plan pricing.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Automation uses Celery for image generation and may be triggered by scheduled runs (frequency/time in `AutomationConfig`).
|
|
||||||
- Other long-running tasks (publishing, optimization, sync) are handled via their Celery tasks/adapters in respective service modules.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Automation enforces exclusivity per site and accounts for credit sufficiency before starting.
|
|
||||||
- Logging (AutomationLogger) produces per-run artifacts and structured traces for observability.
|
|
||||||
- Credit handling is centralized to keep ledger and usage logs consistent and atomic.
|
|
||||||
- Integration services abstract platform handling and allow per-platform test logic.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Reuse `AutomationService` for running/continuing automation; respect locks and stage APIs.
|
|
||||||
- Use `CreditService` before AI/content-heavy operations to check/deduct/add credits and to log usage.
|
|
||||||
- Create invoices via `InvoiceService` helpers rather than constructing manually; update invoice/payment status through service methods.
|
|
||||||
- Manage integrations through `IntegrationService` (create/update/test/list) and extend platform-specific tests as needed.
|
|
||||||
- For domain-specific flows (planner/content/publishing/linking/optimization), place orchestration in the existing service modules and keep tenant context explicit.
|
|
||||||
@@ -1,91 +0,0 @@
|
|||||||
# Automation Module Reference
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Document how the automation module orchestrates multi-stage AI pipelines, exposes API endpoints, enforces tenancy/credits, and manages runs, configs, and logging.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Models: `backend/igny8_core/business/automation/models.py`
|
|
||||||
- Services: `backend/igny8_core/business/automation/services/automation_service.py`, `automation_logger.py`
|
|
||||||
- Tasks (Celery): `backend/igny8_core/business/automation/tasks.py`
|
|
||||||
- API views and routing: `backend/igny8_core/business/automation/views.py`, `urls.py`
|
|
||||||
- Supporting AI functions: `backend/igny8_core/ai/functions/auto_cluster.py`, `generate_ideas.py`, `generate_content.py`, `generate_image_prompts.py`, image queue in `backend/igny8_core/ai/tasks.py`
|
|
||||||
- Tenancy/auth context: `backend/igny8_core/auth/middleware.py`, `backend/igny8_core/api/base.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Maintain per-site automation configs (batch sizes, delays, schedule, enable flag) and track run state with detailed per-stage results.
|
|
||||||
- Provide APIs to configure, trigger, pause/resume/cancel, inspect, and log automation runs.
|
|
||||||
- Execute seven sequential stages that transform planner/writer data via AI and local operations, with credit checks and pause/cancel handling.
|
|
||||||
- Enforce tenant/site scoping on all automation resources and API operations.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- `AutomationConfig` stores enablement, frequency, scheduled time, batch sizes for stages 1–6, and within/between-stage delays. Config is created lazily per site.
|
|
||||||
- `AutomationRun` captures run metadata: trigger type (manual/scheduled), status (`running/paused/cancelled/completed/failed`), current stage, pause/cancel timestamps, per-stage JSON results, total credits used, and error message.
|
|
||||||
- `AutomationService` orchestrates the pipeline:
|
|
||||||
- Locks per site via cache (`automation_lock_{site.id}`) to prevent concurrent runs.
|
|
||||||
- Estimates credits before start and requires a 20% buffer over the estimate against `Account.credits`.
|
|
||||||
- Creates `AutomationRun` with generated `run_id` and logs start via `AutomationLogger`.
|
|
||||||
- Executes stages in order; each stage logs start/progress/complete, applies within/between-stage delays from config, and writes stage result JSON (counts, credits, timestamps, partial flags).
|
|
||||||
- Pause/cancel checks occur inside loops; state is persisted so resumed runs continue from the recorded stage.
|
|
||||||
- Stage credit usage is derived from AI task logs difference before/after the stage.
|
|
||||||
- API layer (`AutomationViewSet`):
|
|
||||||
- `config`/`update_config` read/write `AutomationConfig` for a given `site_id` (scoped to the user’s account).
|
|
||||||
- `run_now` triggers `AutomationService.start_automation` and enqueues Celery `run_automation_task`.
|
|
||||||
- `current_run`, `history`, `logs`, `current_processing`, `estimate`, `pipeline_overview` expose run status, history, logs, credit estimates, and per-stage pending counts.
|
|
||||||
- `pause`, `resume`, `cancel` endpoints update run status and enqueue resume tasks when needed.
|
|
||||||
- Celery tasks:
|
|
||||||
- `check_scheduled_automations` scans enabled configs hourly and triggers runs when frequency/time matches and no recent run exists.
|
|
||||||
- `run_automation_task` performs full pipeline execution.
|
|
||||||
- `resume_automation_task`/`continue_automation_task` continue a paused run from its recorded stage.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `AutomationConfig`, `AutomationRun` (automation state).
|
|
||||||
- Planner models: `Keywords`, `Clusters`, `ContentIdeas`.
|
|
||||||
- Writer models: `Tasks`, `Content`, `Images`.
|
|
||||||
- AI task log (`AITaskLog`) for credit usage measurement.
|
|
||||||
- Tenancy entities: `Account`, `Site` (scoping every query).
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- API call → DRF auth → tenant/site resolved → viewset method → `AutomationService` operations → Celery task (for long-running execution).
|
|
||||||
- Pipeline stages run in-process inside Celery workers, reading planner/writer data, invoking AI functions, updating models, logging progress, and writing stage results to `AutomationRun`.
|
|
||||||
- Completion (or failure) updates run status and releases the site lock.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Planner/writer models supply inputs and receive outputs (clusters, ideas, tasks, content, images).
|
|
||||||
- AI engine executes clustering, idea generation, content generation, and image prompt generation; image rendering uses the AI image queue.
|
|
||||||
- Billing credits are checked against `Account.credits`; credit usage is inferred from AI task logs (deduction logic handled in billing services when those AI calls occur).
|
|
||||||
- Integration/publishing modules consume content/images produced downstream (outside automation).
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Run status moves through `running` → (`paused`/`cancelled`/`failed`/`completed`); `current_stage` increments after each stage finishes; partial flags and timestamps mark mid-stage exits.
|
|
||||||
- Config changes take effect on the next run; pause/resume toggles update run timestamps.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Start blocks if a run is already active for the site or cache lock is held.
|
|
||||||
- Stage loops log and continue on per-batch/item errors; pause/cancel results are persisted mid-stage.
|
|
||||||
- Failures in Celery run mark `AutomationRun` as failed, store error message, timestamp completion, and release the lock.
|
|
||||||
- API endpoints return 400 for missing params or invalid state transitions, 404 for unknown runs, 500 on unexpected errors.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All automation queries filter by `site` tied to the authenticated user’s `account`; config/run creation sets `account` and `site` explicitly.
|
|
||||||
- API endpoints fetch `Site` with `account=request.user.account`; automation locks are per site.
|
|
||||||
- No cross-tenant access; privileged role bypass is handled by DRF auth/permissions upstream.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- Start requires `Account.credits` ≥ 1.2× estimated credits; otherwise a 400 is returned.
|
|
||||||
- Credits actually deducted by AI tasks are reflected via AI task logs and billing services (outside this module); automation aggregates usage per stage in `AutomationRun`.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- Hourly `check_scheduled_automations` respects config frequency/time and last run; skips if a run is already active.
|
|
||||||
- Pipeline execution and resume steps run inside Celery tasks; within-stage sleeps apply delays from config.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Single-run-per-site enforced via cache lock to prevent overlapping credit use or data contention.
|
|
||||||
- Pause/resume/cancel is cooperative, checked inside stage loops, with partial results persisted.
|
|
||||||
- Stage-by-stage logging and result JSON make pipeline progress observable and resumable.
|
|
||||||
- Configurable batch sizes and delays balance throughput and API/credit usage.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Use `AutomationService.start_automation` for new runs; never bypass the cache lock or credit check.
|
|
||||||
- When extending stages, preserve pause/cancel checks, result recording, and credit delta calculation.
|
|
||||||
- Add new API actions through `AutomationViewSet` if they manipulate automation state; keep site/account scoping.
|
|
||||||
- For new schedulers, reuse the lock pattern and `AutomationConfig` fields, and update `next_run_at` appropriately.
|
|
||||||
@@ -1,102 +0,0 @@
|
|||||||
# Automation Pipeline Stages
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Detail the seven pipeline stages executed by `AutomationService`, including inputs, queries, validations, delays, credit handling, and state recording.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Orchestration: `backend/igny8_core/business/automation/services/automation_service.py`
|
|
||||||
- Models: `backend/igny8_core/business/automation/models.py`
|
|
||||||
- AI functions: `backend/igny8_core/ai/functions/auto_cluster.py`, `generate_ideas.py`, `generate_content.py`, `generate_image_prompts.py`
|
|
||||||
- Image queue: `backend/igny8_core/ai/tasks.py` (`process_image_generation_queue`)
|
|
||||||
- Stage entrypoints: `backend/igny8_core/business/automation/tasks.py` (Celery `run_automation_task`, `resume_automation_task`)
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Execute a fixed seven-stage sequence that moves data from planner keywords through content with images and into manual review readiness.
|
|
||||||
- Enforce batch sizes/delays from `AutomationConfig`, support pause/cancel, and write per-stage results into `AutomationRun`.
|
|
||||||
- Track credit deltas per stage using AI task log counts.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
Across all stages:
|
|
||||||
- Each stage logs start/progress/complete via `AutomationLogger`, respects `within_stage_delay` between batches/items, and `between_stage_delay` between stages.
|
|
||||||
- Pause/cancel is checked inside loops; on pause/cancel, the stage records partial counts, credits, elapsed time, and reason, then exits.
|
|
||||||
- Credits used per stage are computed from `AITaskLog` count delta relative to stage start.
|
|
||||||
|
|
||||||
### Stage 1: Keywords → Clusters (AI)
|
|
||||||
- Input query: `Keywords` where `site=current`, `status='new'`, `cluster__isnull=True`, `disabled=False`.
|
|
||||||
- Validation: `validate_minimum_keywords` requires at least 5 keywords; if not valid, stage is skipped with result noting skip reason and `current_stage` advances to 2.
|
|
||||||
- Processing: Batch size = `stage_1_batch_size` (capped to total). For each batch, calls `AIEngine.execute(AutoClusterFunction, payload={'ids': batch})`; waits on task ID; logs per-batch progress. Errors are logged and skipped; pipeline continues.
|
|
||||||
- Result: counts keywords processed, clusters created since run start, batches, credits used, time elapsed; sets `current_stage=2`.
|
|
||||||
|
|
||||||
### Stage 2: Clusters → Ideas (AI)
|
|
||||||
- Pre-check: warns if any `Keywords` still pending from Stage 1.
|
|
||||||
- Input query: `Clusters` where `site=current`, `status='new'`, `disabled=False`.
|
|
||||||
- Processing: Iterates clusters one-by-one; for each, calls `AIEngine.execute(GenerateIdeasFunction, payload={'cluster_id': cluster.id})`; waits on task ID; logs progress. Errors are logged and skipped.
|
|
||||||
- Result: counts clusters processed, ideas created since run start, credits used, time elapsed; sets `current_stage=3`.
|
|
||||||
|
|
||||||
### Stage 3: Ideas → Tasks (Local)
|
|
||||||
- Pre-check: warns if clusters remain without ideas.
|
|
||||||
- Input query: `ContentIdeas` where `site=current`, `status='new'`.
|
|
||||||
- Processing: Batched by `stage_3_batch_size`. For each idea, builds keyword string (M2M keywords or `target_keywords`) and creates `Tasks` with queued status, copying account/site/sector, cluster, content type/structure, and description. Idea status set to `queued`.
|
|
||||||
- Result: ideas processed, tasks created, batches, time elapsed (credits 0 because local); sets `current_stage=4`.
|
|
||||||
|
|
||||||
### Stage 4: Tasks → Content (AI)
|
|
||||||
- Pre-check: warns if `ContentIdeas` remain `new`.
|
|
||||||
- Input query: `Tasks` where `site=current`, `status='queued'`.
|
|
||||||
- Processing: Batched by `stage_4_batch_size`. Uses `GenerateContentFunction` via `AIEngine` per batch (payload contains task IDs). Waits on task IDs, logs progress, continues on errors. Tracks total words by summing generated content word_count.
|
|
||||||
- Result: tasks processed, content created count, total_words, credits used, time elapsed; sets `current_stage=5`.
|
|
||||||
|
|
||||||
### Stage 5: Content → Image Prompts (AI)
|
|
||||||
- Input query: `Content` where `site=current`, `status='draft'`, with zero images (annotated count=0).
|
|
||||||
- Processing: Batched by `stage_5_batch_size`. For each batch, calls `GenerateImagePromptsFunction` via `AIEngine` (payload content IDs). Waits on task IDs, logs progress; continues on errors.
|
|
||||||
- Result: content processed, prompts created (from AI task logs), credits used, time elapsed; sets `current_stage=6`.
|
|
||||||
|
|
||||||
### Stage 6: Image Prompts → Images (AI image queue)
|
|
||||||
- Input query: `Images` where `site=current`, `status='pending'`.
|
|
||||||
- Processing: Iterates pending images; for each, enqueues `process_image_generation_queue.delay(image_ids=[id], account_id, content_id)` when Celery is available, or calls directly in sync fallback. Waits on task IDs with continue-on-error to avoid blocking the stage. Logs progress per image; applies within-stage delay between images.
|
|
||||||
- Result: images processed, images generated (status `generated` since run start), content moved to `review`, credits used, time elapsed; sets `current_stage=7`.
|
|
||||||
|
|
||||||
### Stage 7: Manual Review Gate (Count-only)
|
|
||||||
- Input query: `Content` where `site=current`, `status='review'`.
|
|
||||||
- Processing: Counts review-ready content, logs IDs (truncated), marks run `status='completed'`, sets `completed_at`, and releases the site lock.
|
|
||||||
- Result: ready_for_review count and content IDs stored in `stage_7_result`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Celery task `run_automation_task` instantiates `AutomationService.from_run_id` and calls stages 1→7 sequentially.
|
|
||||||
- Stage transitions update `AutomationRun.current_stage`; between-stage delays applied via `between_stage_delay`.
|
|
||||||
- Resume path (`resume_automation_task`) starts from the recorded `current_stage` and continues through remaining stages.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Planner: Stage 1/2 use `Keywords`/`Clusters`; Stage 3 converts `ContentIdeas` into `Tasks`.
|
|
||||||
- Writer: Stages 4–6 create `Content` and `Images` and move content toward review.
|
|
||||||
- AI engine and functions are invoked in Stages 1, 2, 4, 5; Stage 6 uses the AI image queue.
|
|
||||||
- Billing: Credits are consumed by AI calls; automation records deltas per stage from AI task logs.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- `AutomationRun.status` moves to `completed` at Stage 7; can be set to `failed` on exceptions or `cancelled` via API; `paused` can be set mid-run and resumed.
|
|
||||||
- `current_stage` increments after each successful stage; partial stage results include a `partial` flag and stop reason.
|
|
||||||
- Domain models change status along the pipeline (`Keywords` → clusters, `Clusters` → ideas, `ContentIdeas` → queued/tasks, `Tasks` → completed/content, `Content` → draft/review, `Images` → generated).
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Each stage logs errors and continues to next batch/item; pause/cancel checks short-circuit with partial results saved.
|
|
||||||
- Task wait helper tolerates Celery backend errors; can continue on error when flagged.
|
|
||||||
- Stage start may be skipped with explicit skip reason (e.g., insufficient keywords).
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All queries filter by `site` (and implicit account via tenancy bases); account/site set on created `Tasks` and inherited on `Images` and other records through model save hooks.
|
|
||||||
- Locks and runs are per site; API scoping requires the authenticated user’s account to own the site.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- Start requires sufficient credits (1.2× estimate). Credits used are inferred from AI task log counts per stage; actual deductions occur in AI/billing services invoked by the AI functions.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- Entire stage chain runs inside Celery workers; within-stage sleeps respect config delays; between-stage sleeps applied after each stage.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Idempotent, resume-capable progression with partial state persisted in `AutomationRun`.
|
|
||||||
- Configurable batch sizes/delays mitigate rate limits and manage credit burn.
|
|
||||||
- Continue-on-error semantics prevent single failures from stopping the pipeline while still recording issues.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- When modifying stages, keep pause/cancel checks, stage result recording, and credit delta calculation.
|
|
||||||
- Add new AI stages by wiring through `AIEngine.execute` and the task wait helper; ensure queries are site-scoped and statuses updated.
|
|
||||||
- For new items types, add pending queries and status transitions consistent with existing patterns.
|
|
||||||
@@ -1,75 +0,0 @@
|
|||||||
# Automation Scheduler
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe how scheduled runs are detected, triggered, and resumed using Celery tasks and automation configs.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Celery tasks: `backend/igny8_core/business/automation/tasks.py`
|
|
||||||
- Models: `backend/igny8_core/business/automation/models.py`
|
|
||||||
- Service invoked: `backend/igny8_core/business/automation/services/automation_service.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Periodically scan enabled automation configs to start scheduled runs.
|
|
||||||
- Prevent overlapping runs per site via cache locks and active run checks.
|
|
||||||
- Resume paused runs from their recorded stage.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- `check_scheduled_automations` (Celery, hourly):
|
|
||||||
- Iterates `AutomationConfig` with `is_enabled=True`.
|
|
||||||
- Frequency rules:
|
|
||||||
- `daily`: run when current hour matches `scheduled_time.hour`.
|
|
||||||
- `weekly`: run Mondays at the scheduled hour.
|
|
||||||
- `monthly`: run on the 1st of the month at the scheduled hour.
|
|
||||||
- Skips if `last_run_at` is within ~23 hours or if an `AutomationRun` with `status='running'` exists for the site.
|
|
||||||
- On trigger: instantiates `AutomationService(account, site)`, calls `start_automation(trigger_type='scheduled')`, updates `last_run_at` and `next_run_at` (via `_calculate_next_run`), saves config, and enqueues `run_automation_task.delay(run_id)`.
|
|
||||||
- Exceptions are logged per site; lock release is handled by the service on failure paths.
|
|
||||||
- `run_automation_task`:
|
|
||||||
- Loads service via `from_run_id`, runs stages 1–7 sequentially.
|
|
||||||
- On exception: marks run failed, records error/completed_at, and deletes site lock.
|
|
||||||
- `resume_automation_task` / alias `continue_automation_task`:
|
|
||||||
- Loads service via `from_run_id`, uses `current_stage` to continue remaining stages.
|
|
||||||
- On exception: marks run failed, records error/completed_at.
|
|
||||||
- `_calculate_next_run`:
|
|
||||||
- Computes next run datetime based on frequency and `scheduled_time`, resetting seconds/microseconds; handles month rollover for monthly frequency.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `AutomationConfig`: contains schedule fields (`frequency`, `scheduled_time`, `last_run_at`, `next_run_at`, `is_enabled`).
|
|
||||||
- `AutomationRun`: records run status/stage used during resume/failure handling.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
1) Celery beat (or cron) invokes `check_scheduled_automations` hourly.
|
|
||||||
2) Eligible configs spawn new runs via `AutomationService.start_automation` (includes lock + credit check).
|
|
||||||
3) `run_automation_task` executes the pipeline asynchronously.
|
|
||||||
4) Paused runs can be resumed by enqueueing `resume_automation_task`/`continue_automation_task`, which restart at `current_stage`.
|
|
||||||
5) Failures set run status to `failed` and release locks.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Uses planner/writer data inside the pipeline (see pipeline doc); billing/credits enforced at start.
|
|
||||||
- Locking is done via Django cache, independent of other modules but prevents concurrent Celery runs per site.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Config timestamps (`last_run_at`, `next_run_at`) update on scheduled launch.
|
|
||||||
- Run status changes to `failed` on task exceptions; to `completed` at stage 7; to `paused/cancelled` via API.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Scheduled start is skipped with log messages if recently run or already running.
|
|
||||||
- Exceptions during run execution mark the run failed, record error message, set `completed_at`, and release the cache lock.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Configs and runs are site- and account-scoped; scheduler uses stored account/site from the config; no cross-tenant scheduling.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- Start uses `AutomationService.start_automation`, which enforces credit sufficiency before scheduling the Celery execution.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- Hourly `check_scheduled_automations` plus the long-running `run_automation_task` and resume tasks run in Celery workers.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Hourly scan with coarse matching keeps implementation simple while honoring per-site schedules.
|
|
||||||
- Cache lock and active-run checks prevent double-starts from overlapping schedules or manual triggers.
|
|
||||||
- Resume task reuses the same stage methods to keep behavior consistent between fresh and resumed runs.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- When adding new frequencies, extend `check_scheduled_automations` and `_calculate_next_run` consistently.
|
|
||||||
- Ensure Celery beat (or an equivalent scheduler) runs `check_scheduled_automations` hourly in production.
|
|
||||||
- Preserve lock acquisition and failure handling when modifying task flows to avoid orphaned locks.
|
|
||||||
@@ -1,98 +0,0 @@
|
|||||||
# Billing Module Reference
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Explain how billing is implemented: invoices, payments, credit packages, credit history/usage, payment methods, and limits, as exposed by billing services and API viewsets.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Models: `backend/igny8_core/business/billing/models.py`
|
|
||||||
- Services: `backend/igny8_core/business/billing/services/credit_service.py`, `invoice_service.py`, `payment_service.py`
|
|
||||||
- API (business billing): `backend/igny8_core/business/billing/views.py`, `backend/igny8_core/business/billing/urls.py`
|
|
||||||
- API (core billing endpoints): `backend/igny8_core/modules/billing/views.py`, `backend/igny8_core/modules/billing/urls.py`
|
|
||||||
- Plan metadata (included credits/limits): `backend/igny8_core/auth/models.py` (`Plan`, `Account`)
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Maintain credit ledger, usage logs, configurable per-operation costs, invoices, payments, credit packages, country-level payment method configs, and account-scoped payment methods.
|
|
||||||
- Expose endpoints for invoices/payments/credit packages/transactions, credit balance/usage/limits, and admin billing operations.
|
|
||||||
- Deduct or add credits atomically while recording both transactions and usage logs.
|
|
||||||
- Provide manual and (placeholder) Stripe/PayPal payment flows.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Invoices (`InvoiceViewSet`):
|
|
||||||
- `list`/`retrieve`: fetch account-scoped invoices; include totals, line items, billing period, dates.
|
|
||||||
- `download_pdf`: returns PDF bytes from `InvoiceService.generate_pdf` (currently placeholder text payload).
|
|
||||||
- Payments (`PaymentViewSet`):
|
|
||||||
- `list`: account-scoped payments with amounts, methods, status, invoice linkage, timestamps.
|
|
||||||
- `available_methods`: delegates to `PaymentService.get_available_payment_methods` (country/config-driven; returns methods plus metadata).
|
|
||||||
- `create_manual_payment`: creates a manual payment for an invoice (bank/local wallet); requires invoice_id, method, reference; rejects already-paid invoices; returns pending-approval status.
|
|
||||||
- Account payment methods (`AccountPaymentMethodViewSet`):
|
|
||||||
- CRUD for account-level payment metadata (non-sensitive). `perform_create`/`perform_update` enforce one default by resetting others when `is_default` true or none exists.
|
|
||||||
- `set_default` toggles default within the account.
|
|
||||||
- Credit packages (`CreditPackageViewSet`):
|
|
||||||
- `list`: active packages with credits/price/discount/featured/sort order.
|
|
||||||
- `purchase`: creates invoice via `InvoiceService.create_credit_package_invoice`; returns next action depending on requested payment method (`stripe`/`paypal` placeholders, manual fallback returns invoice info).
|
|
||||||
- Credit transactions (`CreditTransactionViewSet` in business billing views; also registered under `credits/transactions` in URLs): lists credit ledger entries per account.
|
|
||||||
- Core billing endpoints (`modules/billing/views.py`):
|
|
||||||
- `CreditBalanceViewSet.list`: returns current credits, plan monthly credits, credits used this month.
|
|
||||||
- `CreditUsageViewSet`: paginated credit usage logs with filters (operation_type, date range); `summary` aggregates by operation/model and totals (credits, USD); `limits` returns plan/account limits and usage (users/sites/etc.) based on `Plan` fields.
|
|
||||||
- Routing (`business/billing/urls.py`):
|
|
||||||
- Routers for invoices, payments, credit packages, transactions, payment methods, and canonical credit endpoints (balance/usage/transactions). Legacy available-methods endpoint exposed at `/payment-methods/available/`.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `CreditTransaction`: ledger with type, amount (positive/negative), balance_after, metadata.
|
|
||||||
- `CreditUsageLog`: per-AI-operation usage with operation_type, credits_used, cost_usd, model_used, tokens, related object references, metadata.
|
|
||||||
- `CreditCostConfig`: per-operation cost config with unit, display metadata, active flag, audit of previous cost.
|
|
||||||
- `Invoice`: invoice_number, status, amounts (subtotal/tax/total), currency, billing_period, line_items JSON, subscription link, payment metadata, timestamps.
|
|
||||||
- `Payment`: status lifecycle, method (stripe/paypal/bank/local wallet/manual), provider references, manual notes/approval, failure reason, metadata.
|
|
||||||
- `CreditPackage`, `PaymentMethodConfig` (country/method availability + instructions/bank/local wallet data), `AccountPaymentMethod` (account-scoped payment metadata).
|
|
||||||
- Plan/account credits and limits: `Plan.included_credits`, `Plan.max_users/sites/industries/author_profiles`, `Account.credits`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Credits:
|
|
||||||
- Operations call `CreditService.get_credit_cost` → `check_credits`/`deduct_credits` or `deduct_credits_for_operation` → updates `Account.credits`, writes `CreditTransaction` and `CreditUsageLog`.
|
|
||||||
- Costs resolved from `CreditCostConfig` when active; otherwise constants. Units support per request, per 100/200 words, per item/image/idea.
|
|
||||||
- Invoicing/Payments:
|
|
||||||
- Credit package purchase → invoice creation → next-action response (manual vs pending Stripe/PayPal integration).
|
|
||||||
- Manual payment submission → `PaymentService.create_manual_payment` persists pending approval.
|
|
||||||
- Credit balance/usage:
|
|
||||||
- Balance endpoint computes plan credits and month-to-date usage (from `CreditUsageLog`).
|
|
||||||
- Usage endpoint filters logs; summary aggregates by operation/model and totals; limits endpoint computes plan-based limits and usage (users/sites/etc.) per account.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- AI/automation/planner/writer flows trigger credit checks/deductions through `CreditService`; usage logs can be filtered by operation_type (clustering, idea_generation, content_generation, image_generation, optimization, reparse).
|
|
||||||
- Plan limits surfaced in billing limits endpoint affect account/user/site management elsewhere.
|
|
||||||
- Invoice/payment metadata may be used by frontend billing UI; manual payments require admin follow-up.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Credits mutate `Account.credits`; ledger captures each change; usage logs accumulate per operation.
|
|
||||||
- Invoices move through statuses (`draft/pending/paid/void/uncollectible`); payments through (`pending/pending_approval/processing/succeeded/completed/failed/refunded/cancelled`).
|
|
||||||
- Credit packages and payment methods have active/default flags; cost configs can be toggled active/inactive.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Insufficient credits raise `InsufficientCreditsError`; API returns 402 when caught (e.g., content generation, credit deduct flows).
|
|
||||||
- Manual payment rejects missing fields or already-paid invoices.
|
|
||||||
- Usage/balance endpoints return empty data when account/plan missing instead of hard errors.
|
|
||||||
- Validation errors on site/sector/account scoping propagate from base viewsets and model constraints.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Account scoping enforced via `AccountModelViewSet` or explicit account filtering in viewsets; all billing data is per-account.
|
|
||||||
- Payment methods, invoices, payments, transactions, usage logs, and balances are filtered to `request.user.account` (or `request.account` from middleware).
|
|
||||||
- Plan data comes from the account’s associated plan; no cross-tenant visibility.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- Credit costs configurable via `CreditCostConfig`; fallback constants apply when no config.
|
|
||||||
- Credit deductions and ledger entries are atomic; usage logs record the same operation with optional model/token metadata.
|
|
||||||
- Balance endpoint includes plan monthly credits; limits endpoint reports plan/user/site limits but does not enforce them here.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- None specific to billing in this module; Stripe/PayPal integrations are placeholders. Any future webhooks should persist to these models and adjust credits/invoices/payments accordingly.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Credit handling is centralized in `CreditService` to keep ledger and usage logs consistent.
|
|
||||||
- Usage/balance endpoints are read-mostly, with graceful handling when plan/account data is missing.
|
|
||||||
- Manual payments support bank/local wallet flows without storing sensitive data; account payment methods store display metadata only.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Use `CreditService` for all credit checks/deductions/additions; do not mutate `Account.credits` directly.
|
|
||||||
- When adding new AI operations, add `operation_type` to `CreditUsageLog.OPERATION_TYPE_CHOICES` and `CreditCostConfig` seed/constants, then use `deduct_credits_for_operation`.
|
|
||||||
- Extend payment flows by implementing Stripe/PayPal handlers in `PaymentService` and wiring webhooks to update `Payment`/`Invoice`.
|
|
||||||
- Use existing serializers/viewsets when exposing billing data; keep queries scoped to `request.account`.
|
|
||||||
@@ -1,80 +0,0 @@
|
|||||||
# Credit System
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Detail how credits are priced, checked, deducted, logged, and reported across the platform.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Credit logic: `backend/igny8_core/business/billing/services/credit_service.py`
|
|
||||||
- Credit costs config: `backend/igny8_core/business/billing/models.py` (`CreditCostConfig`)
|
|
||||||
- Ledger and usage logs: `backend/igny8_core/business/billing/models.py` (`CreditTransaction`, `CreditUsageLog`)
|
|
||||||
- API endpoints: `backend/igny8_core/modules/billing/views.py` (`CreditBalanceViewSet`, `CreditUsageViewSet`)
|
|
||||||
- Constants and exceptions: `backend/igny8_core/business/billing/constants.py`, `exceptions.py`
|
|
||||||
- Plan included credits: `backend/igny8_core/auth/models.py` (`Plan.included_credits`, `Account.credits`)
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Compute operation costs (configurable per operation and unit) and enforce sufficient balance before AI/content actions.
|
|
||||||
- Deduct or add credits atomically while writing both transaction and usage log records.
|
|
||||||
- Provide balance, usage history, summaries, and limits endpoints for clients.
|
|
||||||
- Allow admin-configurable cost overrides via `CreditCostConfig`.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Cost resolution (`CreditService.get_credit_cost`):
|
|
||||||
- First checks `CreditCostConfig` for the operation (active records). Supports units: `per_request`, `per_100_words`, `per_200_words`, `per_item`, `per_image`. Applies amounts when provided (e.g., words, items, images).
|
|
||||||
- Falls back to hardcoded `CREDIT_COSTS` constants; raises `CreditCalculationError` for unknown operations.
|
|
||||||
- Legacy variable costs: content_generation per 100 words, optimization per 200 words, image_generation per image, idea_generation per idea.
|
|
||||||
- Balance enforcement:
|
|
||||||
- `check_credits`/`check_credits_legacy` raise `InsufficientCreditsError` when balance insufficient.
|
|
||||||
- `deduct_credits_for_operation` computes cost, checks balance, builds a default description per operation when not supplied, then calls `deduct_credits`.
|
|
||||||
- Deduction (`deduct_credits`):
|
|
||||||
- Atomic: decrements `Account.credits`, writes `CreditTransaction` with negative amount and balance_after, writes `CreditUsageLog` capturing operation_type, cost_usd/model/tokens/related object references, metadata.
|
|
||||||
- Add credits (`add_credits`):
|
|
||||||
- Atomic: increments `Account.credits`, writes `CreditTransaction` with positive amount and balance_after.
|
|
||||||
- Logging:
|
|
||||||
- `CreditTransaction` is the authoritative ledger; `CreditUsageLog` tracks per-operation usage for analytics and summaries.
|
|
||||||
- `CreditCostConfig` tracks `previous_cost` on save when cost changes.
|
|
||||||
- Reporting endpoints (modules/billing):
|
|
||||||
- `CreditBalanceViewSet.list`: returns current credits, plan monthly credits, credits used this month (from `CreditUsageLog`), and remaining credits.
|
|
||||||
- `CreditUsageViewSet.list`: paginated usage logs filtered by operation_type and date range, account-scoped.
|
|
||||||
- `CreditUsageViewSet.summary`: aggregates credits and USD cost by operation and model over a date range (defaults to current month).
|
|
||||||
- `CreditUsageViewSet.limits`: returns plan limits (users/sites/industries/author profiles) and current usage counts, plus credits info; returns empty limits if account/plan missing.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `CreditCostConfig`: operation_type, credits_cost, unit, display_name, description, is_active, previous_cost, updated_by, timestamps.
|
|
||||||
- `CreditTransaction`: transaction_type (purchase/subscription/refund/deduction/adjustment), amount (+/-), balance_after, description, metadata, reference_id, created_at.
|
|
||||||
- `CreditUsageLog`: operation_type (clustering/idea_generation/content_generation/image_generation/reparse/legacy names), credits_used, cost_usd, model_used, tokens_in/out, related object type/id, metadata, created_at.
|
|
||||||
- `Account.credits`: current balance; `Plan.included_credits`: monthly included credits.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Caller requests an operation → `get_credit_cost` computes cost → `check_credits` ensures balance → `deduct_credits` mutates balance and writes ledger + usage → upstream operation proceeds (AI/content).
|
|
||||||
- Balance/usage endpoints read from `Account`, `CreditUsageLog`, `Plan` to present current balances, month-to-date usage, summaries, and limits.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- AI/automation/planner/writer services call `CreditService` before expensive operations; automation aggregates credits used per stage from AI task logs, while billing logs capture actual deductions at AI call sites.
|
|
||||||
- Plan limits (users/sites/industries/authors) referenced in billing limits endpoint inform account management elsewhere.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Balance changes recorded per transaction; usage accumulates per operation. Cost configs can be toggled active/inactive and updated with audit of previous cost.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Unknown operation types raise `CreditCalculationError`; insufficient balance raises `InsufficientCreditsError` (mapped to HTTP 402 in callers).
|
|
||||||
- Balance/usage endpoints fall back to zeros/empty when account/plan missing rather than erroring.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All credit operations and queries are account-scoped; viewsets filter by `request.account`/`request.user.account`. No cross-tenant access.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- Costs derived from `CreditCostConfig` > constants; units must match supplied amount. Ledger and usage logs are mandatory for every deduction/addition.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- None dedicated; credit usage often originates from AI Celery tasks but logging happens at the service call site.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Centralized credit math and atomic DB updates prevent drift between balance, ledger, and usage logs.
|
|
||||||
- Configurable costs allow runtime tuning without code changes.
|
|
||||||
- Reporting endpoints are read-only and tolerant of missing plan data to keep dashboards resilient.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Always route credit checks/deductions/additions through `CreditService`.
|
|
||||||
- When adding new AI operations, register operation_type in `CreditUsageLog`/constants and optionally seed `CreditCostConfig`.
|
|
||||||
- Include amount (words/items/images) when calling `deduct_credits_for_operation` so unit-based pricing applies.
|
|
||||||
- Use balance/usage endpoints for UI/analytics; avoid custom queries that bypass account scoping.
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,80 +0,0 @@
|
|||||||
# Planner Overview
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Explain how the planner module manages keywords, clusters, and content ideas, including tenancy, validation, and API behaviors.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Models: `backend/igny8_core/business/planning/models.py`
|
|
||||||
- Serializers: `backend/igny8_core/modules/planner/serializers.py`
|
|
||||||
- Views: `backend/igny8_core/modules/planner/views.py`
|
|
||||||
- URLs: `backend/igny8_core/modules/planner/urls.py`
|
|
||||||
- Services: `backend/igny8_core/business/planning/services/clustering_service.py`, `ideas_service.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Store site/sector-scoped keywords, clusters, and content ideas linked to global seed keywords.
|
|
||||||
- Provide CRUD and bulk operations for keywords/clusters/ideas with search, filter, and ordering.
|
|
||||||
- Validate industry/sector alignment between seed keywords and site/sector.
|
|
||||||
- Feed downstream writer/automation stages with structured ideas and clusters.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Models (site/sector scoped via `SiteSectorBaseModel` and soft-delete):
|
|
||||||
- `Keywords`: references global `SeedKeyword`; optional volume/difficulty overrides; optional cluster; status `new/mapped`; disabled flag; validation ensures seed keyword industry/sector match the site/sector.
|
|
||||||
- `Clusters`: topic groups with counts, volume, mapped_pages, status `new/mapped`; unique per site/sector by name.
|
|
||||||
- `ContentIdeas`: ideas tied to clusters and optional keyword objects; status `new/queued/completed`; content_type/structure and estimated_word_count; disabled flag.
|
|
||||||
- Serializers:
|
|
||||||
- `KeywordSerializer` enforces `seed_keyword_id` on create; exposes seed keyword-derived fields (keyword/volume/difficulty/intent) read-only; optional overrides; includes site_id/sector_id write-only; provides cluster/sector names via getters; optional `attribute_values` when `USE_SITE_BUILDER_REFACTOR` flag is enabled.
|
|
||||||
- `ClusterSerializer` exposes site/sector IDs, sector name, read-only counts/volume/mapped_pages.
|
|
||||||
- `ContentIdeasSerializer` exposes cluster/sector names, content type/structure, keyword/target fields, site/sector write-only.
|
|
||||||
- Views (`KeywordViewSet`, `ClusterViewSet`, `ContentIdeasViewSet`):
|
|
||||||
- Inherit `SiteSectorModelViewSet` for tenant/site/sector filtering and unified responses.
|
|
||||||
- Permissions: `IsAuthenticatedAndActive` + viewer-or-above (search/list CRUD).
|
|
||||||
- Pagination: `CustomPageNumberPagination`; throttle scope `planner`.
|
|
||||||
- Filters/Search/Ordering: keywords searchable by seed_keyword.keyword; filter by status, cluster_id, seed intent; ordering by created_at, volume, difficulty; custom range filters for difficulty/volume; clusters filter by status; ideas filter by status/cluster.
|
|
||||||
- Creation requires explicit `site_id` and `sector_id`; viewsets validate site/sector existence and alignment; set `account/site/sector` explicitly on save.
|
|
||||||
- Bulk endpoints:
|
|
||||||
- Keywords: `bulk_delete`, two definitions of `bulk_update` (status update) present; `bulk_add_from_seed` to create Keywords from SeedKeywords with site/sector validation.
|
|
||||||
- Clusters: bulk update endpoints in view (not shown above) adjust status.
|
|
||||||
- Ideas: creation and status updates follow standard CRUD; not shown as bulk endpoints in code snippet.
|
|
||||||
- Error handling: viewsets wrap errors into unified responses; return 400/404/500 as appropriate.
|
|
||||||
- Services:
|
|
||||||
- `ClusteringService` and `IdeasService` (not detailed here) provide additional business logic for clustering/idea generation when invoked by automation or endpoints.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `Keywords`, `Clusters`, `ContentIdeas`; global `SeedKeyword`.
|
|
||||||
- Tenancy entities: `Account`, `Site`, `Sector`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Requests → DRF auth → `SiteSectorModelViewSet` filters by account/site/sector → serializers validate seed/site/sector alignment → models persisted → responses returned via unified helpers; bulk actions operate on filtered querysets.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Automation stages 1–3 consume planner data (keywords → clusters → ideas → tasks).
|
|
||||||
- Writer tasks reference clusters/ideas produced here.
|
|
||||||
- Billing credits may be checked upstream for AI clustering/idea generation (via services invoked in automation or planner actions).
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Keywords: `new` → `mapped`; Clusters: `new` → `mapped`; ContentIdeas: `new` → `queued` → `completed`.
|
|
||||||
- Disabled flag can exclude records from processes.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Validation enforces seed/site/sector industry alignment; missing site/sector on create raises validation errors.
|
|
||||||
- Bulk operations return 400 on missing IDs/status; general errors return unified 500 responses.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All planner models inherit `SiteSectorBaseModel`; queries are filtered by account/site/sector; admin/developer/system roles can bypass filtering via base viewset logic.
|
|
||||||
- Create requires explicit site and sector; viewsets fetch and set account from site/user/middleware.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- None directly in the planner endpoints; credit costs for clustering/idea generation are enforced where those services are called (automation or planner service usage).
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- Planner endpoints run synchronously; automation may batch planner data into AI tasks via Celery (documented in automation).
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Strict site/sector validation prevents cross-tenant or cross-site leaks.
|
|
||||||
- Seed keyword linkage centralizes keyword metadata; overrides allow per-site tuning.
|
|
||||||
- Pagination/filtering/search are optimized for large keyword sets.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Use provided viewsets and serializers; ensure site_id/sector_id are supplied on create.
|
|
||||||
- When adding fields, extend serializers with read-only/write-only behavior consistent with current patterns.
|
|
||||||
- For new bulk operations, follow existing patterns: validate IDs, operate on filtered queryset, return unified responses.
|
|
||||||
@@ -1,82 +0,0 @@
|
|||||||
# Writer Overview
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe how the writer module manages tasks, content, images, and taxonomies, including validations, tenancy, and API behaviors.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Models: `backend/igny8_core/business/content/models.py`
|
|
||||||
- Serializers: `backend/igny8_core/modules/writer/serializers.py`
|
|
||||||
- Views: `backend/igny8_core/modules/writer/views.py`
|
|
||||||
- URLs: `backend/igny8_core/modules/writer/urls.py`
|
|
||||||
- Services: `backend/igny8_core/business/content/services/content_generation_service.py`, `validation_service.py`, `metadata_mapping_service.py`
|
|
||||||
- Automation linkages: `backend/igny8_core/business/automation/services/automation_service.py` (stages 3–6)
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Manage tenant/site/sector-scoped tasks, generated/imported content, images, and taxonomies.
|
|
||||||
- Provide CRUD and bulk operations for tasks and images; content generation triggers; taxonomy CRUD.
|
|
||||||
- Enforce site/sector alignment on creation and propagate account/site/sector to related records.
|
|
||||||
- Feed automation and publishing flows with content and images, and integrate with billing credits for AI operations.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Models (all inherit tenant/site/sector bases; many soft-deletable):
|
|
||||||
- `Tasks`: queued work items tied to clusters/ideas/taxonomy; content type/structure; keywords text; word_count target; status `queued/completed`.
|
|
||||||
- `Content`: generated/imported content with HTML, SEO fields, cluster link, content type/structure, taxonomy M2M, external IDs/URLs/metadata, sync status, source (`igny8/wordpress`), status (`draft/review/published`), word_count, timestamps.
|
|
||||||
- `Images`: linked to content or task; auto-sets account/site/sector from provider; tracks type, URL/path, prompt, status, position.
|
|
||||||
- `ContentTaxonomy`: site/sector taxonomy terms (category/tag) with external taxonomy/id, sync status, description, count, metadata; unique per site by slug+type and by external id+taxonomy.
|
|
||||||
- Supporting: `ContentClusterMap`, `ContentAttribute` (documented in domain models).
|
|
||||||
- Serializers:
|
|
||||||
- `TasksSerializer`: requires cluster/content_type/content_structure on create; exposes cluster/sector names; write-only site_id/sector_id; defaults status to queued if absent.
|
|
||||||
- `ImagesSerializer`: exposes task/content titles; read-only derived fields; creation requires site/sector (validated, with fallback to request user active site default sector); sets account/site/sector via base perform_create.
|
|
||||||
- `ContentSerializer`: exposes cluster/sector names; taxonomy terms grouped (tags/categories); image status flags; write-only site_id/sector_id; validates presence of required fields and status transitions.
|
|
||||||
- Group serializers for images support grouped responses by content.
|
|
||||||
- Views:
|
|
||||||
- `TasksViewSet`: filters/search/order; bulk_delete, bulk_update status; `auto_generate_content` triggers `ContentGenerationService.generate_content` for up to 10 tasks, enforcing account presence and credit checks (returns 402 on `InsufficientCreditsError`); inherits tenant/site/sector filtering.
|
|
||||||
- `ImagesViewSet`: CRUD with ordering/filtering; perform_create enforces site/sector and populates context; `serve_image_file` serves local files with basic checks.
|
|
||||||
- `ContentViewSet`: CRUD for content; ordering/filtering; handles taxonomy assignments; exposes status/SEO fields; uses base scoping.
|
|
||||||
- `ContentTaxonomyViewSet`: CRUD for taxonomy terms (category/tag) scoped to site/sector; inherits base scoping.
|
|
||||||
- Services:
|
|
||||||
- `ContentGenerationService`: invoked by `auto_generate_content`; orchestrates AI content generation for tasks, returns async task_id when queued or synchronous result; raises `InsufficientCreditsError` on low balance.
|
|
||||||
- `ContentValidationService`, `MetadataMappingService`: additional processing/validation/mapping (used in writer flows and publishing).
|
|
||||||
- Automation linkage:
|
|
||||||
- Stage 3 creates tasks from ideas; Stage 4 generates content; Stage 5 generates image prompts; Stage 6 generates images and moves content to review.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `Tasks`, `Content`, `Images`, `ContentTaxonomy`, plus `ContentClusterMap`/`ContentAttribute`.
|
|
||||||
- Tenancy entities: `Account`, `Site`, `Sector`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Requests → DRF auth → `SiteSectorModelViewSet` filtering → serializer validation (site/sector required on create) → model save → unified response.
|
|
||||||
- `auto_generate_content` validates task IDs/account, calls service → service may enqueue Celery AI tasks or run synchronously → response with task_id or result.
|
|
||||||
- Images creation auto-inherits account/site/sector from linked task/content; local file serving bypasses account filter for read-only access.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Planner clusters/ideas feed tasks; tasks/content/images feed publishing and automation stages.
|
|
||||||
- Billing credits enforced via `ContentGenerationService` when AI is used; automation aggregates credits used per stage from AI task logs.
|
|
||||||
- Integration/publishing sync uses content/taxonomy/image data when pushing to external platforms.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Tasks: `queued` → `completed`; Content: `draft` → `review` → `published`; Images: `pending` → `generated`; Taxonomies created/updated/deleted via CRUD.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Viewsets return unified errors; bulk operations validate IDs/status; `auto_generate_content` returns 400/404/402/500 depending on validation, missing tasks, insufficient credits, or unexpected errors.
|
|
||||||
- Images file serving returns 404 on missing file/image.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Base viewset filters enforce account/site/sector scoping; admins/developers/system can bypass filtering; persisted records retain tenant/site/sector ownership.
|
|
||||||
- Creation requires site/sector; account set from request/site/user; Images perform_create validates presence of site/sector.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- AI content generation is guarded by credit checks; InsufficientCredits returns HTTP 402; credit logging occurs in billing services at AI call sites.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- AI generation may run via Celery when `ContentGenerationService` returns a task_id; automation stages run inside Celery workers for bulk flows.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Strict site/sector requirements prevent cross-site contamination.
|
|
||||||
- Bulk limits (max 10 tasks in auto_generate_content) to control credit burn and processing load.
|
|
||||||
- Image serving permits public access via endpoint while still performing existence checks.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Use existing serializers/viewsets; keep site/sector required on create.
|
|
||||||
- When extending AI generation, ensure billing checks (`CreditService`) and account/site scoping are preserved.
|
|
||||||
- For new bulk actions, follow current patterns: validate input, operate on filtered queryset, return unified responses.
|
|
||||||
290
docs/10-MODULES/AUTOMATION.md
Normal file
290
docs/10-MODULES/AUTOMATION.md
Normal file
@@ -0,0 +1,290 @@
|
|||||||
|
# Automation Module
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Status:** ✅ Active
|
||||||
|
**Backend Path:** `backend/igny8_core/business/automation/`
|
||||||
|
**Frontend Path:** `frontend/src/pages/Automation/`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
|
||||||
|
| What | File | Key Items |
|
||||||
|
|------|------|-----------|
|
||||||
|
| Models | `business/automation/models.py` | `AutomationConfig`, `AutomationRun` |
|
||||||
|
| Service | `business/automation/services/automation_service.py` | `AutomationService` |
|
||||||
|
| Logger | `business/automation/services/automation_logger.py` | `AutomationLogger` |
|
||||||
|
| Celery Tasks | `business/automation/tasks.py` | `run_automation_task`, `check_scheduled_automations` |
|
||||||
|
| Frontend | `pages/Automation/AutomationPage.tsx` | Main automation UI |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
The Automation module runs the complete 7-stage content pipeline automatically:
|
||||||
|
|
||||||
|
```
|
||||||
|
Keywords → Clusters → Ideas → Tasks → Content → Image Prompts → Images → Review
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7-Stage Pipeline
|
||||||
|
|
||||||
|
| Stage | Name | AI Function | Credit Cost |
|
||||||
|
|-------|------|-------------|-------------|
|
||||||
|
| 1 | Keywords → Clusters | `AutoClusterFunction` | Per batch |
|
||||||
|
| 2 | Clusters → Ideas | `GenerateIdeasFunction` | Per idea |
|
||||||
|
| 3 | Ideas → Tasks | None (local) | None |
|
||||||
|
| 4 | Tasks → Content | `GenerateContentFunction` | Per 100 words |
|
||||||
|
| 5 | Content → Image Prompts | `GenerateImagePromptsFunction` | Per prompt |
|
||||||
|
| 6 | Image Prompts → Images | `process_image_generation_queue` | Per image |
|
||||||
|
| 7 | Manual Review Gate | None | None |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Data Models
|
||||||
|
|
||||||
|
### AutomationConfig
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| site | FK | Target site |
|
||||||
|
| enabled | Boolean | Enable/disable automation |
|
||||||
|
| frequency | CharField | daily/weekly/monthly |
|
||||||
|
| scheduled_time | TimeField | Time to run |
|
||||||
|
| stage_1_batch_size | Integer | Keywords per batch |
|
||||||
|
| stage_2_batch_size | Integer | Clusters per batch |
|
||||||
|
| stage_3_batch_size | Integer | Ideas per batch |
|
||||||
|
| stage_4_batch_size | Integer | Tasks per batch |
|
||||||
|
| stage_5_batch_size | Integer | Content per batch |
|
||||||
|
| stage_6_batch_size | Integer | Images per batch |
|
||||||
|
| within_stage_delay | Integer | Seconds between batches |
|
||||||
|
| between_stage_delay | Integer | Seconds between stages |
|
||||||
|
| last_run_at | DateTime | Last execution |
|
||||||
|
| next_run_at | DateTime | Next scheduled run |
|
||||||
|
|
||||||
|
### AutomationRun
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| config | FK | Parent config |
|
||||||
|
| trigger_type | CharField | manual/scheduled |
|
||||||
|
| status | CharField | running/paused/cancelled/completed/failed |
|
||||||
|
| current_stage | Integer | Current stage (1-7) |
|
||||||
|
| started_at | DateTime | Start time |
|
||||||
|
| paused_at | DateTime | Pause time (nullable) |
|
||||||
|
| resumed_at | DateTime | Resume time (nullable) |
|
||||||
|
| cancelled_at | DateTime | Cancel time (nullable) |
|
||||||
|
| completed_at | DateTime | Completion time (nullable) |
|
||||||
|
| total_credits_used | Decimal | Total credits consumed |
|
||||||
|
| stage_1_result | JSON | Stage 1 results |
|
||||||
|
| stage_2_result | JSON | Stage 2 results |
|
||||||
|
| stage_3_result | JSON | Stage 3 results |
|
||||||
|
| stage_4_result | JSON | Stage 4 results |
|
||||||
|
| stage_5_result | JSON | Stage 5 results |
|
||||||
|
| stage_6_result | JSON | Stage 6 results |
|
||||||
|
| stage_7_result | JSON | Stage 7 results |
|
||||||
|
| error_message | TextField | Error details (nullable) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/api/v1/automation/config/` | Get/create config | Get automation config |
|
||||||
|
| PUT | `/api/v1/automation/update_config/` | Update config | Update settings |
|
||||||
|
| POST | `/api/v1/automation/run_now/` | Start manual run | Start automation |
|
||||||
|
| GET | `/api/v1/automation/current_run/` | Get current run | Run status/progress |
|
||||||
|
| GET | `/api/v1/automation/pipeline_overview/` | Get pipeline | Stage status counts |
|
||||||
|
| GET | `/api/v1/automation/current_processing/` | Get processing | Live processing status |
|
||||||
|
| POST | `/api/v1/automation/pause/` | Pause run | Pause after current item |
|
||||||
|
| POST | `/api/v1/automation/resume/` | Resume run | Resume from saved stage |
|
||||||
|
| POST | `/api/v1/automation/cancel/` | Cancel run | Cancel after current item |
|
||||||
|
| GET | `/api/v1/automation/history/` | Get history | Last 20 runs |
|
||||||
|
| GET | `/api/v1/automation/logs/` | Get logs | Activity log for run |
|
||||||
|
| GET | `/api/v1/automation/estimate/` | Get estimate | Credit estimate |
|
||||||
|
|
||||||
|
**Query Parameters:** All require `?site_id=`, run-specific require `?run_id=`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Execution Flow
|
||||||
|
|
||||||
|
### Manual Run
|
||||||
|
|
||||||
|
1. User clicks "Run Now" on frontend
|
||||||
|
2. Frontend calls `POST /automation/run_now/?site_id=X`
|
||||||
|
3. Backend acquires cache lock `automation_lock_{site_id}`
|
||||||
|
4. Estimates credits required (1.2x buffer)
|
||||||
|
5. Validates balance >= estimate
|
||||||
|
6. Creates `AutomationRun` record
|
||||||
|
7. Enqueues `run_automation_task` Celery task
|
||||||
|
8. Returns run ID immediately
|
||||||
|
|
||||||
|
### Stage Execution
|
||||||
|
|
||||||
|
For each stage (1-7):
|
||||||
|
|
||||||
|
1. Check `_check_should_stop()` (paused/cancelled?)
|
||||||
|
2. Load items for processing
|
||||||
|
3. Process in batches (respecting batch_size)
|
||||||
|
4. For AI stages: Call AIEngine function
|
||||||
|
5. Wait `within_stage_delay` between batches
|
||||||
|
6. Save stage result JSON
|
||||||
|
7. Wait `between_stage_delay` before next stage
|
||||||
|
|
||||||
|
### Stage Result Fields
|
||||||
|
|
||||||
|
**Stage 1 (Clustering):**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"keywords_processed": 150,
|
||||||
|
"clusters_created": 12,
|
||||||
|
"batches_run": 3,
|
||||||
|
"credits_used": 45,
|
||||||
|
"time_elapsed": 120,
|
||||||
|
"skipped": false,
|
||||||
|
"partial": false
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Stage 2 (Ideas):**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"clusters_processed": 12,
|
||||||
|
"ideas_created": 36,
|
||||||
|
"batches_run": 2,
|
||||||
|
"credits_used": 72
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Stage 3 (Tasks):**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"ideas_processed": 36,
|
||||||
|
"tasks_created": 36,
|
||||||
|
"batches_run": 4
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Stage 4 (Content):**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"tasks_processed": 36,
|
||||||
|
"content_created": 36,
|
||||||
|
"total_words": 54000,
|
||||||
|
"batches_run": 6,
|
||||||
|
"credits_used": 540
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Stage 5 (Image Prompts):**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"content_processed": 36,
|
||||||
|
"prompts_created": 180,
|
||||||
|
"batches_run": 4,
|
||||||
|
"credits_used": 36
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Stage 6 (Images):**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"images_processed": 180,
|
||||||
|
"images_generated": 180,
|
||||||
|
"batches_run": 18
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Stage 7 (Review):**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"ready_for_review": 36
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Scheduling
|
||||||
|
|
||||||
|
**Celery Beat Task:** `check_scheduled_automations`
|
||||||
|
**Frequency:** Hourly
|
||||||
|
|
||||||
|
**Logic:**
|
||||||
|
1. Find configs where `enabled=True`
|
||||||
|
2. Check if `next_run_at <= now`
|
||||||
|
3. Check if no active run exists
|
||||||
|
4. Start `run_automation_task` for eligible configs
|
||||||
|
5. Update `next_run_at` based on frequency
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Lock Mechanism
|
||||||
|
|
||||||
|
**Purpose:** Prevent concurrent runs for same site
|
||||||
|
|
||||||
|
**Key:** `automation_lock_{site_id}`
|
||||||
|
**Storage:** Redis cache
|
||||||
|
**Acquired:** On run start
|
||||||
|
**Released:** On completion/failure/cancel
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Credit Validation
|
||||||
|
|
||||||
|
Before starting:
|
||||||
|
1. Calculate estimated credits for all stages
|
||||||
|
2. Apply 1.2x safety buffer
|
||||||
|
3. Compare with account balance
|
||||||
|
4. Reject if balance < estimate
|
||||||
|
|
||||||
|
During execution:
|
||||||
|
- Each AI stage checks credits before processing
|
||||||
|
- Deductions happen after successful AI calls
|
||||||
|
- `total_credits_used` accumulates across stages
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Frontend Integration
|
||||||
|
|
||||||
|
### AutomationPage Components
|
||||||
|
|
||||||
|
- **Config Panel:** Enable/disable, schedule settings
|
||||||
|
- **Pipeline Cards:** Stage-by-stage status with pending counts
|
||||||
|
- **Processing Card:** Live processing status during run
|
||||||
|
- **Control Buttons:** Run Now, Pause, Resume, Cancel
|
||||||
|
- **Activity Log:** Real-time log streaming
|
||||||
|
- **History Table:** Past 20 runs with status
|
||||||
|
|
||||||
|
### Polling
|
||||||
|
|
||||||
|
- Every ~5s while run is running/paused
|
||||||
|
- Fetches: `current_run`, `pipeline_overview`, `current_processing`
|
||||||
|
- Lighter polling when idle
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
| Issue | Cause | Fix |
|
||||||
|
|-------|-------|-----|
|
||||||
|
| "Already running" error | Lock exists from previous run | Wait or check if stuck |
|
||||||
|
| Insufficient credits | Balance < 1.2x estimate | Add credits |
|
||||||
|
| Stage skipped | No items to process | Check previous stages |
|
||||||
|
| Run stuck | Worker crashed | Clear lock, restart |
|
||||||
|
| Images not generating | Stage 5 didn't create prompts | Check stage 5 result |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
|
||||||
|
| Feature | Status | Description |
|
||||||
|
|---------|--------|-------------|
|
||||||
|
| Progress bar accuracy | 🐛 Bug | Fix progress calculation based on actual stage |
|
||||||
|
| Completed count display | 🐛 Bug | Fix count display in UI |
|
||||||
|
| Stage skip configuration | 🔜 Planned | Allow skipping certain stages |
|
||||||
|
| Notification on complete | 🔜 Planned | Email/webhook when done |
|
||||||
254
docs/10-MODULES/BILLING.md
Normal file
254
docs/10-MODULES/BILLING.md
Normal file
@@ -0,0 +1,254 @@
|
|||||||
|
# Billing Module
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Status:** ✅ Active
|
||||||
|
**Backend Path:** `backend/igny8_core/modules/billing/` + `backend/igny8_core/business/billing/`
|
||||||
|
**Frontend Path:** `frontend/src/pages/Billing/` + `frontend/src/pages/Account/`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
|
||||||
|
| What | File | Key Items |
|
||||||
|
|------|------|-----------|
|
||||||
|
| Models | `business/billing/models.py` | `CreditTransaction`, `CreditUsageLog`, `CreditCostConfig` |
|
||||||
|
| Service | `business/billing/services/credit_service.py` | `CreditService` |
|
||||||
|
| Views | `modules/billing/views.py` | `CreditBalanceViewSet`, `CreditUsageViewSet` |
|
||||||
|
| Frontend | `pages/Account/PlansAndBillingPage.tsx` | Plans, credits, billing history |
|
||||||
|
| Store | `store/billingStore.ts` | `useBillingStore` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
The Billing module manages:
|
||||||
|
- Credit balance and transactions
|
||||||
|
- Usage tracking and limits
|
||||||
|
- Plan enforcement
|
||||||
|
- Payment processing
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Data Models
|
||||||
|
|
||||||
|
### CreditTransaction (Ledger)
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| transaction_type | CharField | purchase/subscription/refund/deduction/adjustment |
|
||||||
|
| amount | Decimal | Positive (add) or negative (deduct) |
|
||||||
|
| balance_after | Decimal | Balance after transaction |
|
||||||
|
| description | CharField | Transaction description |
|
||||||
|
| metadata | JSON | Additional data |
|
||||||
|
| reference_id | CharField | External reference |
|
||||||
|
| created_at | DateTime | Transaction time |
|
||||||
|
|
||||||
|
### CreditUsageLog (Analytics)
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| operation_type | CharField | clustering/idea_generation/content_generation/image_generation |
|
||||||
|
| credits_used | Decimal | Credits consumed |
|
||||||
|
| cost_usd | Decimal | USD cost |
|
||||||
|
| model_used | CharField | AI model used |
|
||||||
|
| tokens_in | Integer | Input tokens |
|
||||||
|
| tokens_out | Integer | Output tokens |
|
||||||
|
| content_type | FK | Related content type |
|
||||||
|
| object_id | Integer | Related object ID |
|
||||||
|
| metadata | JSON | Additional data |
|
||||||
|
| created_at | DateTime | Usage time |
|
||||||
|
|
||||||
|
### CreditCostConfig
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| operation_type | CharField | Operation name |
|
||||||
|
| credits_cost | Decimal | Credits per unit |
|
||||||
|
| unit | CharField | per_request/per_100_words/per_200_words/per_item/per_image |
|
||||||
|
| display_name | CharField | UI display name |
|
||||||
|
| description | TextField | Description |
|
||||||
|
| is_active | Boolean | Enable/disable |
|
||||||
|
| previous_cost | Decimal | Previous cost (audit) |
|
||||||
|
| updated_by | FK | Last updater |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Credit Calculation Methods
|
||||||
|
|
||||||
|
### Method 1: Token-Based (Text Models)
|
||||||
|
|
||||||
|
Used for: Clustering, Ideas, Content Generation, Optimization
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. AI call completes
|
||||||
|
2. OpenAI returns actual token usage
|
||||||
|
3. Calculate: `total_tokens = input_tokens + output_tokens`
|
||||||
|
4. Look up `CreditCostConfig` for operation
|
||||||
|
5. Apply formula based on unit:
|
||||||
|
- `per_100_words`: `credits = ceil(words / 100) * cost`
|
||||||
|
- `per_item`: `credits = items * cost`
|
||||||
|
6. Apply minimum credits
|
||||||
|
7. Deduct from balance
|
||||||
|
|
||||||
|
### Method 2: Fixed Cost (Image Models)
|
||||||
|
|
||||||
|
Used for: Image Generation
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. Before AI call
|
||||||
|
2. Calculate: `credits = num_images * cost_per_image`
|
||||||
|
3. Check balance sufficient
|
||||||
|
4. Deduct credits
|
||||||
|
5. Then make AI call
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/api/v1/billing/balance/` | `CreditBalanceViewSet.list` | Current balance + monthly usage |
|
||||||
|
| GET | `/api/v1/billing/usage/` | `CreditUsageViewSet.list` | Usage log (paginated) |
|
||||||
|
| GET | `/api/v1/billing/usage/summary/` | `CreditUsageViewSet.summary` | Aggregated by operation |
|
||||||
|
| GET | `/api/v1/billing/usage/limits/` | `CreditUsageViewSet.limits` | Plan limits + current usage |
|
||||||
|
| GET | `/api/v1/billing/transactions/` | `TransactionViewSet.list` | Transaction history |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Credit Service API
|
||||||
|
|
||||||
|
### Check Credits
|
||||||
|
|
||||||
|
```python
|
||||||
|
CreditService.check_credits(account, required_credits)
|
||||||
|
# Raises InsufficientCreditsError if balance < required
|
||||||
|
```
|
||||||
|
|
||||||
|
### Deduct Credits
|
||||||
|
|
||||||
|
```python
|
||||||
|
CreditService.deduct_credits_for_operation(
|
||||||
|
account=account,
|
||||||
|
operation_type='content_generation',
|
||||||
|
amount=word_count, # For per_word operations
|
||||||
|
model='gpt-4o-mini',
|
||||||
|
tokens_in=2500,
|
||||||
|
tokens_out=1500,
|
||||||
|
metadata={'content_id': 123}
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Add Credits
|
||||||
|
|
||||||
|
```python
|
||||||
|
CreditService.add_credits(
|
||||||
|
account=account,
|
||||||
|
amount=1000,
|
||||||
|
transaction_type='purchase',
|
||||||
|
description='Credit package purchase'
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Plan Limits
|
||||||
|
|
||||||
|
### Hard Limits (Never Reset)
|
||||||
|
|
||||||
|
| Limit | Field | Description |
|
||||||
|
|-------|-------|-------------|
|
||||||
|
| Sites | `max_sites` | Maximum sites per account |
|
||||||
|
| Users | `max_users` | Maximum team members |
|
||||||
|
| Keywords | `max_keywords` | Total keywords allowed |
|
||||||
|
| Clusters | `max_clusters` | Total clusters allowed |
|
||||||
|
|
||||||
|
### Monthly Limits (Reset on Billing Cycle)
|
||||||
|
|
||||||
|
| Limit | Field | Description |
|
||||||
|
|-------|-------|-------------|
|
||||||
|
| Content Ideas | `max_content_ideas` | Ideas per month |
|
||||||
|
| Content Words | `max_content_words` | Words generated per month |
|
||||||
|
| Basic Images | `max_images_basic` | Basic AI images per month |
|
||||||
|
| Premium Images | `max_images_premium` | Premium AI images per month |
|
||||||
|
| Image Prompts | `max_image_prompts` | Prompts per month |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Usage Limits Panel
|
||||||
|
|
||||||
|
**Component:** `UsageLimitsPanel.tsx`
|
||||||
|
|
||||||
|
Displays:
|
||||||
|
- Progress bars for each limit
|
||||||
|
- Color coding: blue (safe), yellow (warning), red (critical)
|
||||||
|
- Days until reset for monthly limits
|
||||||
|
- Upgrade CTA when approaching limits
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Credit Costs Reference
|
||||||
|
|
||||||
|
Current costs (from `CreditCostConfig`):
|
||||||
|
|
||||||
|
| Operation | Unit | Default Cost |
|
||||||
|
|-----------|------|--------------|
|
||||||
|
| Clustering | per_request | 10 credits |
|
||||||
|
| Idea Generation | per_item | 2 credits |
|
||||||
|
| Content Generation | per_100_words | 5 credits |
|
||||||
|
| Image Generation (Basic) | per_image | 3 credits |
|
||||||
|
| Image Generation (Premium) | per_image | 5 credits |
|
||||||
|
| Content Optimization | per_200_words | 3 credits |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Frontend Pages
|
||||||
|
|
||||||
|
### Plans & Billing (`/account/plans`)
|
||||||
|
|
||||||
|
**Tabs:**
|
||||||
|
1. **Current Plan** - Active plan, upgrade options
|
||||||
|
2. **Credits Overview** - Balance, usage chart, cost breakdown
|
||||||
|
3. **Purchase Credits** - Credit packages
|
||||||
|
4. **Billing History** - Invoices and transactions
|
||||||
|
|
||||||
|
### Usage Analytics (`/account/usage`)
|
||||||
|
|
||||||
|
**Tabs:**
|
||||||
|
1. **Limits & Usage** - Plan limits with progress bars
|
||||||
|
2. **Activity** - Credit transaction history
|
||||||
|
3. **API Usage** - API call statistics
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Integration Points
|
||||||
|
|
||||||
|
| From | To | Trigger |
|
||||||
|
|------|----|---------|
|
||||||
|
| AIEngine | CreditService | Pre-check and post-deduct |
|
||||||
|
| Automation | CreditService | Per-stage credit tracking |
|
||||||
|
| Subscription | Account | Credit allocation |
|
||||||
|
| Admin | CreditCostConfig | Price adjustments |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
| Issue | Cause | Fix |
|
||||||
|
|-------|-------|-----|
|
||||||
|
| 402 error | Insufficient credits | Add credits or reduce operation |
|
||||||
|
| Usage not showing | Log not created | Check CreditService called |
|
||||||
|
| Limits wrong | Cache stale | Clear cache, reload |
|
||||||
|
| Monthly usage high | Automation running | Pause automation |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
|
||||||
|
| Feature | Status | Description |
|
||||||
|
|---------|--------|-------------|
|
||||||
|
| AI Model Config database | 🔜 Planned | Move model pricing to database |
|
||||||
|
| Budget alerts | 🔜 Planned | Email when approaching limits |
|
||||||
|
| Usage forecasting | 🔜 Planned | Predict credit usage |
|
||||||
|
| Overage billing | 🔜 Planned | Charge for overages instead of blocking |
|
||||||
286
docs/10-MODULES/INTEGRATIONS.md
Normal file
286
docs/10-MODULES/INTEGRATIONS.md
Normal file
@@ -0,0 +1,286 @@
|
|||||||
|
# Integrations Module
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Status:** ✅ Active
|
||||||
|
**Backend Path:** `backend/igny8_core/modules/integration/` + `backend/igny8_core/business/integration/`
|
||||||
|
**Frontend Path:** `frontend/src/pages/Settings/IntegrationSettings.tsx`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
|
||||||
|
| What | File | Key Items |
|
||||||
|
|------|------|-----------|
|
||||||
|
| Models | `business/integration/models.py` | `SiteIntegration`, `SyncEvent` |
|
||||||
|
| Views | `modules/integration/views.py` | `SiteIntegrationViewSet` |
|
||||||
|
| Webhooks | `modules/integration/webhooks.py` | `wordpress_webhook` |
|
||||||
|
| Services | `business/integration/services/*.py` | Sync services |
|
||||||
|
| Frontend | `pages/Settings/IntegrationSettings.tsx` | Integration UI |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
The Integrations module manages:
|
||||||
|
- WordPress site connections
|
||||||
|
- Two-way content synchronization
|
||||||
|
- Webhook handling
|
||||||
|
- External platform integrations
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Data Models
|
||||||
|
|
||||||
|
### SiteIntegration
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| site | FK | IGNY8 site |
|
||||||
|
| platform | CharField | wordpress/shopify |
|
||||||
|
| site_url | URLField | External site URL |
|
||||||
|
| api_key | CharField | WordPress Application Password |
|
||||||
|
| username | CharField | WordPress username |
|
||||||
|
| is_active | Boolean | Enable/disable |
|
||||||
|
| sync_enabled | Boolean | Enable auto-sync |
|
||||||
|
| last_sync_at | DateTime | Last sync time |
|
||||||
|
| sync_status | CharField | idle/syncing/error |
|
||||||
|
| metadata | JSON | Platform-specific data |
|
||||||
|
| created_at | DateTime | Creation date |
|
||||||
|
|
||||||
|
### SyncEvent
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| integration | FK | Parent integration |
|
||||||
|
| event_type | CharField | content_push/content_pull/webhook |
|
||||||
|
| direction | CharField | igny8_to_wp/wp_to_igny8 |
|
||||||
|
| content_type | CharField | post/page/product |
|
||||||
|
| content_id | Integer | IGNY8 content ID |
|
||||||
|
| external_id | Integer | WordPress post ID |
|
||||||
|
| status | CharField | pending/success/failed |
|
||||||
|
| error_message | TextField | Error details |
|
||||||
|
| metadata | JSON | Additional data |
|
||||||
|
| created_at | DateTime | Event time |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/api/v1/integration/` | `SiteIntegrationViewSet.list` | List integrations |
|
||||||
|
| POST | `/api/v1/integration/` | `SiteIntegrationViewSet.create` | Create integration |
|
||||||
|
| PUT | `/api/v1/integration/{id}/` | `SiteIntegrationViewSet.update` | Update integration |
|
||||||
|
| DELETE | `/api/v1/integration/{id}/` | `SiteIntegrationViewSet.destroy` | Remove integration |
|
||||||
|
| POST | `/api/v1/integration/{id}/test_connection/` | Test connection | Verify credentials |
|
||||||
|
| POST | `/api/v1/integration/{id}/test_collection_connection/` | Test per-collection | Test specific content type |
|
||||||
|
| POST | `/api/v1/integration/{id}/sync/` | Trigger sync | Start synchronization |
|
||||||
|
| GET | `/api/v1/integration/{id}/sync_status/` | Get sync status | Current sync progress |
|
||||||
|
| POST | `/api/v1/integration/{id}/update_structure/` | Update structure | Refresh site structure |
|
||||||
|
| GET | `/api/v1/integration/{id}/content_types/` | Get content types | Available types + counts |
|
||||||
|
| GET | `/api/v1/integration/{id}/sync_health/` | Get sync health | Sync statistics |
|
||||||
|
| POST | `/api/v1/integration/site_sync/` | Site-level sync | Sync by site ID |
|
||||||
|
|
||||||
|
### Webhooks
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| POST | `/api/v1/integration/webhook/wordpress/` | `wordpress_webhook` | Receive WP updates |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## WordPress Integration
|
||||||
|
|
||||||
|
### Setup Flow
|
||||||
|
|
||||||
|
1. User enters WordPress site URL
|
||||||
|
2. User creates Application Password in WordPress
|
||||||
|
3. User enters credentials in IGNY8
|
||||||
|
4. IGNY8 tests connection via WordPress REST API
|
||||||
|
5. If successful, saves `SiteIntegration` record
|
||||||
|
|
||||||
|
### Required WordPress Setup
|
||||||
|
|
||||||
|
- WordPress 5.6+ (REST API enabled)
|
||||||
|
- Application Passwords enabled
|
||||||
|
- Pretty permalinks enabled
|
||||||
|
- REST API accessible (no blocking plugins)
|
||||||
|
|
||||||
|
### Test Connection
|
||||||
|
|
||||||
|
**Endpoint:** `POST /integration/{id}/test_connection/`
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. Call WordPress `/wp-json/wp/v2/users/me`
|
||||||
|
2. Verify authentication works
|
||||||
|
3. Return user info and site capabilities
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Content Synchronization
|
||||||
|
|
||||||
|
### IGNY8 → WordPress (Push)
|
||||||
|
|
||||||
|
**Trigger:** User clicks "Publish to WordPress"
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. Content is in `approved` status
|
||||||
|
2. Get `SiteIntegration` for content's site
|
||||||
|
3. Build WordPress post payload:
|
||||||
|
- title, content, excerpt, slug
|
||||||
|
- status: `publish` or `draft`
|
||||||
|
- categories, tags (mapped)
|
||||||
|
- featured media (if image exists)
|
||||||
|
4. Call WordPress REST API:
|
||||||
|
- `POST /wp-json/wp/v2/posts` (new)
|
||||||
|
- `PUT /wp-json/wp/v2/posts/{id}` (update)
|
||||||
|
5. Store `wordpress_id` on Content
|
||||||
|
6. Create `SyncEvent` record
|
||||||
|
7. Update Content status to `published`
|
||||||
|
|
||||||
|
### WordPress → IGNY8 (Pull)
|
||||||
|
|
||||||
|
**Trigger:** Manual sync or webhook
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. Call WordPress `/wp-json/wp/v2/posts`
|
||||||
|
2. For each post:
|
||||||
|
- Check if exists in IGNY8 by `wordpress_id`
|
||||||
|
- If exists: Update if modified
|
||||||
|
- If new: Create Content record
|
||||||
|
3. Sync taxonomies (categories, tags)
|
||||||
|
4. Create `SyncEvent` records
|
||||||
|
|
||||||
|
### Webhook Handling
|
||||||
|
|
||||||
|
**WordPress Plugin** sends webhooks to IGNY8 when:
|
||||||
|
- Post created/updated/deleted
|
||||||
|
- Post status changed (draft → published)
|
||||||
|
|
||||||
|
**Webhook Payload:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"action": "post_updated",
|
||||||
|
"post_id": 123,
|
||||||
|
"post_type": "post",
|
||||||
|
"site_url": "https://example.com"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**IGNY8 Processing:**
|
||||||
|
1. Validate webhook signature (optional)
|
||||||
|
2. Find matching `SiteIntegration`
|
||||||
|
3. Fetch full post from WordPress
|
||||||
|
4. Update/create Content in IGNY8
|
||||||
|
5. Create `SyncEvent` record
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Image Sync
|
||||||
|
|
||||||
|
### Push (IGNY8 → WordPress)
|
||||||
|
|
||||||
|
1. Image record has `image_url`
|
||||||
|
2. Download image from URL
|
||||||
|
3. Upload to WordPress Media Library
|
||||||
|
4. Get WordPress attachment ID
|
||||||
|
5. Set as featured image on post
|
||||||
|
|
||||||
|
### Pull (WordPress → IGNY8)
|
||||||
|
|
||||||
|
1. Get featured media ID from post
|
||||||
|
2. Fetch media URL from WordPress
|
||||||
|
3. Store URL in IGNY8 Images record
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Taxonomy Sync
|
||||||
|
|
||||||
|
### Categories
|
||||||
|
|
||||||
|
1. Get WordPress categories via `/wp-json/wp/v2/categories`
|
||||||
|
2. Create `ContentTaxonomy` records (type=category)
|
||||||
|
3. Store `wordpress_id` for mapping
|
||||||
|
|
||||||
|
### Tags
|
||||||
|
|
||||||
|
1. Get WordPress tags via `/wp-json/wp/v2/tags`
|
||||||
|
2. Create `ContentTaxonomy` records (type=tag)
|
||||||
|
3. Store `wordpress_id` for mapping
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Sync Status Tracking
|
||||||
|
|
||||||
|
| Status | Description |
|
||||||
|
|--------|-------------|
|
||||||
|
| idle | No sync in progress |
|
||||||
|
| syncing | Sync currently running |
|
||||||
|
| error | Last sync failed |
|
||||||
|
|
||||||
|
**Sync Health Metrics:**
|
||||||
|
- Total synced posts
|
||||||
|
- Last successful sync
|
||||||
|
- Failed sync count
|
||||||
|
- Pending items
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Frontend Pages
|
||||||
|
|
||||||
|
### Integration Settings (`/settings/integration`)
|
||||||
|
|
||||||
|
- Add new integration button
|
||||||
|
- List of configured integrations
|
||||||
|
- Status indicators (connected/error)
|
||||||
|
- Test connection button
|
||||||
|
- Sync now button
|
||||||
|
- Sync status and history
|
||||||
|
|
||||||
|
### WordPress Sync Dashboard (`/sites/{id}/sync`)
|
||||||
|
|
||||||
|
- Detailed sync status
|
||||||
|
- Content sync queue
|
||||||
|
- Error log
|
||||||
|
- Manual sync triggers
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
| Issue | Cause | Fix |
|
||||||
|
|-------|-------|-----|
|
||||||
|
| Connection failed | Invalid credentials | Verify Application Password |
|
||||||
|
| 401 Unauthorized | Password expired | Regenerate Application Password |
|
||||||
|
| 403 Forbidden | REST API blocked | Check security plugins |
|
||||||
|
| Images not syncing | CORS/URL issues | Verify image URLs accessible |
|
||||||
|
| Categories missing | Not synced | Run sync_structure first |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Security
|
||||||
|
|
||||||
|
### Credential Storage
|
||||||
|
|
||||||
|
- API keys encrypted at rest
|
||||||
|
- Never exposed in API responses
|
||||||
|
- Application Passwords rotatable
|
||||||
|
|
||||||
|
### Webhook Security
|
||||||
|
|
||||||
|
- Optional signature verification
|
||||||
|
- Site URL validation
|
||||||
|
- Rate limiting on webhook endpoint
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
|
||||||
|
| Feature | Status | Description |
|
||||||
|
|---------|--------|-------------|
|
||||||
|
| Shopify integration | 🔜 Planned | E-commerce platform support |
|
||||||
|
| Ghost integration | 🔜 Planned | Ghost CMS support |
|
||||||
|
| Webflow integration | 🔜 Planned | Webflow CMS support |
|
||||||
|
| Scheduled sync | 🔜 Planned | Automatic periodic sync |
|
||||||
|
| Conflict resolution | 🔜 Planned | Handle edit conflicts |
|
||||||
183
docs/10-MODULES/LINKER.md
Normal file
183
docs/10-MODULES/LINKER.md
Normal file
@@ -0,0 +1,183 @@
|
|||||||
|
# Linker Module
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Status:** ⏸️ Inactive (Disabled by Default)
|
||||||
|
**Backend Path:** `backend/igny8_core/modules/linker/`
|
||||||
|
**Frontend Path:** `frontend/src/pages/Linker/`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Module Status
|
||||||
|
|
||||||
|
| Aspect | Current State | Notes |
|
||||||
|
|--------|---------------|-------|
|
||||||
|
| Backend API | ✅ Implemented | Endpoints functional |
|
||||||
|
| Frontend Pages | ✅ Implemented | UI exists |
|
||||||
|
| Sidebar Nav | ⚠️ Conditional | Hidden when disabled |
|
||||||
|
| Route Protection | ❌ Not Protected | Direct URL access still works |
|
||||||
|
| Dashboard References | ❌ Not Hidden | May show in dashboard |
|
||||||
|
| Default State | Disabled | `linker_enabled = True` in model, but typically disabled |
|
||||||
|
|
||||||
|
**⚠️ Pending Implementation:** Extend module disable to route-level protection and all page references.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
|
||||||
|
| What | File | Key Items |
|
||||||
|
|------|------|-----------|
|
||||||
|
| Views | `modules/linker/views.py` | `LinkerViewSet` |
|
||||||
|
| API | `modules/linker/urls.py` | Linker endpoints |
|
||||||
|
| Frontend Pages | `pages/Linker/index.tsx` | Linker overview |
|
||||||
|
| Frontend Pages | `pages/Linker/LinkerContent.tsx` | Content for linking |
|
||||||
|
| API Client | `api/linker.api.ts` | `linkerApi` |
|
||||||
|
| Module Control | `modules/system/settings_models.py` | `ModuleEnableSettings.linker_enabled` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
The Linker module provides internal linking automation:
|
||||||
|
- Identify link opportunities in content
|
||||||
|
- Suggest relevant internal links
|
||||||
|
- Auto-inject links into content body
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| POST | `/api/v1/linker/process/` | `LinkerViewSet.process` | Process single content for linking |
|
||||||
|
| POST | `/api/v1/linker/batch_process/` | `LinkerViewSet.batch_process` | Process multiple content items |
|
||||||
|
|
||||||
|
### Process Request
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"content_id": 123,
|
||||||
|
"max_links": 5,
|
||||||
|
"anchor_strategy": "exact_match"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Process Response
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"content_id": 123,
|
||||||
|
"links_added": 3,
|
||||||
|
"link_opportunities": [
|
||||||
|
{
|
||||||
|
"anchor_text": "SEO optimization",
|
||||||
|
"target_url": "/blog/seo-guide/",
|
||||||
|
"target_content_id": 456,
|
||||||
|
"position": 234,
|
||||||
|
"confidence": 0.85
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Business Logic
|
||||||
|
|
||||||
|
### Link Candidate Discovery
|
||||||
|
|
||||||
|
**Trigger:** User initiates linking process
|
||||||
|
**Flow:**
|
||||||
|
1. Load source content body
|
||||||
|
2. Extract potential anchor phrases
|
||||||
|
3. Find matching content by:
|
||||||
|
- Keyword overlap
|
||||||
|
- Title similarity
|
||||||
|
- Same sector/industry
|
||||||
|
4. Score candidates by relevance
|
||||||
|
5. Filter to avoid over-linking
|
||||||
|
|
||||||
|
### Link Injection
|
||||||
|
|
||||||
|
**Trigger:** User approves link suggestions
|
||||||
|
**Flow:**
|
||||||
|
1. Locate anchor text in content
|
||||||
|
2. Wrap in `<a>` tag with target URL
|
||||||
|
3. Ensure no nested links
|
||||||
|
4. Save updated content body
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Module Enable Control
|
||||||
|
|
||||||
|
### Backend Model
|
||||||
|
|
||||||
|
```python
|
||||||
|
# modules/system/settings_models.py
|
||||||
|
class ModuleEnableSettings(AccountBaseModel):
|
||||||
|
linker_enabled = models.BooleanField(default=True)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Frontend Check
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// layout/AppSidebar.tsx
|
||||||
|
if (isModuleEnabled('linker')) {
|
||||||
|
workflowItems.push({
|
||||||
|
name: "Linker",
|
||||||
|
path: "/linker/content",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Current Limitation
|
||||||
|
|
||||||
|
Direct URL access to `/linker/*` routes still works even when module is disabled.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Frontend Pages
|
||||||
|
|
||||||
|
### Linker Overview (`/linker`)
|
||||||
|
|
||||||
|
- Module overview
|
||||||
|
- Quick stats
|
||||||
|
- Start linking action
|
||||||
|
|
||||||
|
### Content for Linking (`/linker/content`)
|
||||||
|
|
||||||
|
- List of content available for linking
|
||||||
|
- Link status indicators
|
||||||
|
- Process button per content
|
||||||
|
- Batch processing action
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Integration Points
|
||||||
|
|
||||||
|
| From | To | Trigger |
|
||||||
|
|------|----|---------|
|
||||||
|
| Content | Linker | Manual process |
|
||||||
|
| Linker | Content | Link injection |
|
||||||
|
| Automation | Linker | Automated linking (future) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
| Issue | Cause | Fix |
|
||||||
|
|-------|-------|-----|
|
||||||
|
| Module visible when disabled | Only sidebar hidden | Pending: route protection |
|
||||||
|
| No link candidates | No matching content | Create more related content |
|
||||||
|
| Links broken | Target content deleted | Validate target exists |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
|
||||||
|
| Feature | Status | Description |
|
||||||
|
|---------|--------|-------------|
|
||||||
|
| Route-level protection | 🔜 Pending | Block access when module disabled |
|
||||||
|
| Dashboard card hiding | 🔜 Pending | Hide from dashboard when disabled |
|
||||||
|
| Automation integration | 🔜 Planned | Add to automation pipeline |
|
||||||
|
| Link density control | 🔜 Planned | Prevent over-linking |
|
||||||
|
| External link support | 🔜 Planned | Add outbound links |
|
||||||
262
docs/10-MODULES/OPTIMIZER.md
Normal file
262
docs/10-MODULES/OPTIMIZER.md
Normal file
@@ -0,0 +1,262 @@
|
|||||||
|
# Optimizer Module
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Status:** ⏸️ Inactive (Disabled by Default)
|
||||||
|
**Backend Path:** `backend/igny8_core/modules/optimizer/` + `backend/igny8_core/business/optimization/`
|
||||||
|
**Frontend Path:** `frontend/src/pages/Optimizer/`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Module Status
|
||||||
|
|
||||||
|
| Aspect | Current State | Notes |
|
||||||
|
|--------|---------------|-------|
|
||||||
|
| Backend API | ✅ Implemented | Endpoints functional |
|
||||||
|
| Business Logic | ✅ Implemented | Optimization service exists |
|
||||||
|
| Frontend Pages | ✅ Implemented | UI exists |
|
||||||
|
| Sidebar Nav | ⚠️ Conditional | Hidden when disabled |
|
||||||
|
| Route Protection | ❌ Not Protected | Direct URL access still works |
|
||||||
|
| Dashboard References | ❌ Not Hidden | May show in dashboard |
|
||||||
|
| Default State | Disabled | `optimizer_enabled = True` in model, but typically disabled |
|
||||||
|
|
||||||
|
**⚠️ Pending Implementation:** Extend module disable to route-level protection and all page references.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
|
||||||
|
| What | File | Key Items |
|
||||||
|
|------|------|-----------|
|
||||||
|
| Views | `modules/optimizer/views.py` | `OptimizerViewSet` |
|
||||||
|
| Models | `business/optimization/models.py` | `OptimizationTask` |
|
||||||
|
| Services | `business/optimization/services/*.py` | Optimization logic |
|
||||||
|
| AI Function | `ai/functions/optimize.py` | `OptimizeContentFunction` |
|
||||||
|
| Frontend Pages | `pages/Optimizer/index.tsx` | Optimizer overview |
|
||||||
|
| Frontend Pages | `pages/Optimizer/OptimizerContent.tsx` | Select content |
|
||||||
|
| Frontend Pages | `pages/Optimizer/OptimizerPreview.tsx` | Analysis preview |
|
||||||
|
| API Client | `api/optimizer.api.ts` | `optimizerApi` |
|
||||||
|
| Module Control | `modules/system/settings_models.py` | `ModuleEnableSettings.optimizer_enabled` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
The Optimizer module provides AI-powered content optimization:
|
||||||
|
- Analyze existing content for SEO improvements
|
||||||
|
- Suggest and apply optimizations
|
||||||
|
- Track before/after scores
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Data Models
|
||||||
|
|
||||||
|
### OptimizationTask
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| site | FK | Parent site |
|
||||||
|
| sector | FK | Parent sector |
|
||||||
|
| content | FK | Target content |
|
||||||
|
| entry_point | CharField | auto/igny8/wordpress/external/manual |
|
||||||
|
| status | CharField | pending/analyzing/optimizing/completed/failed |
|
||||||
|
| original_score | Integer | Score before optimization |
|
||||||
|
| optimized_score | Integer | Score after optimization |
|
||||||
|
| original_content | TextField | Content before changes |
|
||||||
|
| optimized_content | TextField | Content after changes |
|
||||||
|
| suggestions | JSON | Optimization suggestions |
|
||||||
|
| applied_changes | JSON | Changes that were applied |
|
||||||
|
| credits_used | Decimal | Credits consumed |
|
||||||
|
| created_at | DateTime | Creation date |
|
||||||
|
| completed_at | DateTime | Completion date |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| POST | `/api/v1/optimizer/optimize/` | `OptimizerViewSet.optimize` | Optimize single content |
|
||||||
|
| POST | `/api/v1/optimizer/batch_optimize/` | `OptimizerViewSet.batch_optimize` | Optimize multiple items |
|
||||||
|
| POST | `/api/v1/optimizer/analyze/` | `OptimizerViewSet.analyze` | Analyze without applying |
|
||||||
|
|
||||||
|
### Entry Points
|
||||||
|
|
||||||
|
| Entry Point | Description |
|
||||||
|
|-------------|-------------|
|
||||||
|
| `auto` | Auto-detect based on content source |
|
||||||
|
| `igny8` | IGNY8-generated content |
|
||||||
|
| `wordpress` | WordPress-synced content |
|
||||||
|
| `external` | External platform content |
|
||||||
|
| `manual` | Manual optimization trigger |
|
||||||
|
|
||||||
|
### Optimize Request
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"content_id": 123,
|
||||||
|
"entry_point": "igny8",
|
||||||
|
"optimization_goals": ["seo", "readability", "keyword_density"]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Optimize Response
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"task_id": 456,
|
||||||
|
"content_id": 123,
|
||||||
|
"original_score": 65,
|
||||||
|
"optimized_score": 85,
|
||||||
|
"suggestions": [
|
||||||
|
{
|
||||||
|
"type": "keyword_density",
|
||||||
|
"description": "Increase keyword 'SEO' usage",
|
||||||
|
"applied": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "meta_description",
|
||||||
|
"description": "Meta description too short",
|
||||||
|
"applied": true
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"credits_used": 15
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Business Logic
|
||||||
|
|
||||||
|
### Content Analysis
|
||||||
|
|
||||||
|
**Trigger:** User clicks "Analyze"
|
||||||
|
**Credit Cost:** None (analysis only)
|
||||||
|
|
||||||
|
**Checks:**
|
||||||
|
- Keyword density and distribution
|
||||||
|
- Meta title and description optimization
|
||||||
|
- Heading structure (H1, H2, H3)
|
||||||
|
- Readability score
|
||||||
|
- Content length
|
||||||
|
- Internal/external link count
|
||||||
|
- Image alt text presence
|
||||||
|
|
||||||
|
### Content Optimization (AI)
|
||||||
|
|
||||||
|
**Trigger:** User clicks "Optimize"
|
||||||
|
**AI Function:** `OptimizeContentFunction`
|
||||||
|
**Credit Cost:** Per 200 words
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. Load content with metadata
|
||||||
|
2. Analyze current state
|
||||||
|
3. AIEngine executes `OptimizeContentFunction`:
|
||||||
|
- Reviews content against SEO best practices
|
||||||
|
- Suggests improvements
|
||||||
|
- Rewrites sections if needed
|
||||||
|
4. Create `OptimizationTask` record
|
||||||
|
5. Save original and optimized versions
|
||||||
|
6. Return comparison view
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Module Enable Control
|
||||||
|
|
||||||
|
### Backend Model
|
||||||
|
|
||||||
|
```python
|
||||||
|
# modules/system/settings_models.py
|
||||||
|
class ModuleEnableSettings(AccountBaseModel):
|
||||||
|
optimizer_enabled = models.BooleanField(default=True)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Frontend Check
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// layout/AppSidebar.tsx
|
||||||
|
if (isModuleEnabled('optimizer')) {
|
||||||
|
workflowItems.push({
|
||||||
|
name: "Optimizer",
|
||||||
|
path: "/optimizer/content",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Current Limitation
|
||||||
|
|
||||||
|
Direct URL access to `/optimizer/*` routes still works even when module is disabled.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Frontend Pages
|
||||||
|
|
||||||
|
### Optimizer Overview (`/optimizer`)
|
||||||
|
|
||||||
|
- Module overview
|
||||||
|
- Quick stats
|
||||||
|
- Recent optimizations
|
||||||
|
|
||||||
|
### Select Content (`/optimizer/content`)
|
||||||
|
|
||||||
|
- List of content available for optimization
|
||||||
|
- Filter by score, status
|
||||||
|
- Start analysis button
|
||||||
|
- Batch optimize action
|
||||||
|
|
||||||
|
### Analysis Preview (`/optimizer/preview`)
|
||||||
|
|
||||||
|
- Side-by-side comparison
|
||||||
|
- Suggestion list
|
||||||
|
- Accept/reject changes
|
||||||
|
- Apply optimization button
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Optimization Scoring
|
||||||
|
|
||||||
|
| Metric | Weight | Description |
|
||||||
|
|--------|--------|-------------|
|
||||||
|
| Keyword Density | 20% | Target 1-3% density |
|
||||||
|
| Meta Quality | 15% | Title 50-60 chars, desc 150-160 chars |
|
||||||
|
| Heading Structure | 15% | Proper H1-H6 hierarchy |
|
||||||
|
| Readability | 15% | Flesch-Kincaid score |
|
||||||
|
| Content Length | 10% | Meets word count target |
|
||||||
|
| Internal Links | 10% | 2-5 internal links |
|
||||||
|
| Image Optimization | 10% | Alt text, sizing |
|
||||||
|
| Mobile Friendly | 5% | No wide elements |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Integration Points
|
||||||
|
|
||||||
|
| From | To | Trigger |
|
||||||
|
|------|----|---------|
|
||||||
|
| Content | Optimizer | Manual analysis/optimize |
|
||||||
|
| WordPress | Optimizer | Synced content optimization |
|
||||||
|
| Optimizer | Content | Apply changes |
|
||||||
|
| Automation | Optimizer | Automated optimization (future) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
| Issue | Cause | Fix |
|
||||||
|
|-------|-------|-----|
|
||||||
|
| Module visible when disabled | Only sidebar hidden | Pending: route protection |
|
||||||
|
| Optimization not improving | Content already good | Check original score |
|
||||||
|
| High credit usage | Large content | Optimize sections |
|
||||||
|
| Changes not saving | Apply not clicked | Click "Apply Changes" |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
|
||||||
|
| Feature | Status | Description |
|
||||||
|
|---------|--------|-------------|
|
||||||
|
| Route-level protection | 🔜 Pending | Block access when module disabled |
|
||||||
|
| Dashboard card hiding | 🔜 Pending | Hide from dashboard when disabled |
|
||||||
|
| Automation integration | 🔜 Planned | Add to automation pipeline |
|
||||||
|
| Partial optimization | 🔜 Planned | Optimize specific sections only |
|
||||||
|
| Competitor analysis | 🔜 Planned | Compare against top-ranking content |
|
||||||
|
| A/B testing | 🔜 Planned | Track performance of optimizations |
|
||||||
231
docs/10-MODULES/PLANNER.md
Normal file
231
docs/10-MODULES/PLANNER.md
Normal file
@@ -0,0 +1,231 @@
|
|||||||
|
# Planner Module
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Status:** ✅ Active
|
||||||
|
**Backend Path:** `backend/igny8_core/modules/planner/`
|
||||||
|
**Frontend Path:** `frontend/src/pages/Planner/`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
|
||||||
|
| What | File | Key Items |
|
||||||
|
|------|------|-----------|
|
||||||
|
| Models | `modules/planner/models.py` | `Keywords`, `Clusters`, `ContentIdeas` |
|
||||||
|
| Views | `modules/planner/views.py` | `KeywordViewSet`, `ClusterViewSet`, `ContentIdeaViewSet` |
|
||||||
|
| Serializers | `modules/planner/serializers.py` | Keyword/Cluster/Idea serializers |
|
||||||
|
| AI Functions | `ai/functions/clustering.py` | `AutoClusterFunction` |
|
||||||
|
| AI Functions | `ai/functions/ideas.py` | `GenerateIdeasFunction` |
|
||||||
|
| Frontend Pages | `pages/Planner/Keywords.tsx` | Keyword management |
|
||||||
|
| Frontend Pages | `pages/Planner/Clusters.tsx` | Cluster management |
|
||||||
|
| Frontend Pages | `pages/Planner/Ideas.tsx` | Content ideas |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
The Planner module manages the SEO content planning pipeline:
|
||||||
|
|
||||||
|
```
|
||||||
|
SeedKeywords (Global) → Keywords (Site/Sector) → Clusters → ContentIdeas → Tasks
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Data Models
|
||||||
|
|
||||||
|
### Keywords
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| site | FK | Parent site |
|
||||||
|
| sector | FK | Parent sector |
|
||||||
|
| seed_keyword | FK | Reference to global SeedKeyword |
|
||||||
|
| keyword | CharField | Keyword text |
|
||||||
|
| search_volume | Integer | Monthly search volume |
|
||||||
|
| difficulty | Integer | SEO difficulty (0-100) |
|
||||||
|
| cpc | Decimal | Cost per click |
|
||||||
|
| search_intent | CharField | informational/commercial/transactional/navigational |
|
||||||
|
| status | CharField | new/mapped/used |
|
||||||
|
| cluster | FK | Assigned cluster (nullable) |
|
||||||
|
| created_at | DateTime | Creation date |
|
||||||
|
|
||||||
|
### Clusters
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| site | FK | Parent site |
|
||||||
|
| sector | FK | Parent sector |
|
||||||
|
| name | CharField | Cluster name |
|
||||||
|
| description | TextField | Cluster description |
|
||||||
|
| primary_keyword | FK | Main keyword |
|
||||||
|
| status | CharField | draft/active/complete |
|
||||||
|
| keyword_count | Integer | Number of keywords |
|
||||||
|
| created_at | DateTime | Creation date |
|
||||||
|
|
||||||
|
### ContentIdeas
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| site | FK | Parent site |
|
||||||
|
| sector | FK | Parent sector |
|
||||||
|
| cluster | FK | Source cluster |
|
||||||
|
| title | CharField | Content title |
|
||||||
|
| description | TextField | Content brief |
|
||||||
|
| target_keywords | JSON | Keywords to target |
|
||||||
|
| content_type | CharField | blog_post/guide/comparison/etc. |
|
||||||
|
| word_count_target | Integer | Target word count |
|
||||||
|
| status | CharField | draft/queued/used |
|
||||||
|
| priority | Integer | Priority score |
|
||||||
|
| created_at | DateTime | Creation date |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
|
||||||
|
### Keywords
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/api/v1/planner/keywords/` | `KeywordViewSet.list` | List keywords with filtering |
|
||||||
|
| POST | `/api/v1/planner/keywords/` | `KeywordViewSet.create` | Create single keyword |
|
||||||
|
| GET | `/api/v1/planner/keywords/{id}/` | `KeywordViewSet.retrieve` | Get keyword detail |
|
||||||
|
| PUT | `/api/v1/planner/keywords/{id}/` | `KeywordViewSet.update` | Update keyword |
|
||||||
|
| DELETE | `/api/v1/planner/keywords/{id}/` | `KeywordViewSet.destroy` | Soft delete keyword |
|
||||||
|
| POST | `/api/v1/planner/keywords/bulk_delete/` | `KeywordViewSet.bulk_delete` | Hard delete multiple |
|
||||||
|
| POST | `/api/v1/planner/keywords/bulk_status/` | `KeywordViewSet.bulk_status` | Update status for multiple |
|
||||||
|
| POST | `/api/v1/planner/keywords/add_to_workflow/` | `KeywordViewSet.add_to_workflow` | Add SeedKeywords to workflow |
|
||||||
|
|
||||||
|
**Filters:** `?site_id=`, `?sector_id=`, `?status=`, `?cluster_id=`, `?difficulty_min=`, `?difficulty_max=`, `?volume_min=`, `?volume_max=`
|
||||||
|
|
||||||
|
### Clusters
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/api/v1/planner/clusters/` | `ClusterViewSet.list` | List clusters |
|
||||||
|
| POST | `/api/v1/planner/clusters/` | `ClusterViewSet.create` | Create cluster manually |
|
||||||
|
| POST | `/api/v1/planner/clusters/auto_cluster/` | `ClusterViewSet.auto_cluster` | AI-powered clustering |
|
||||||
|
| POST | `/api/v1/planner/clusters/generate_ideas/` | `ClusterViewSet.generate_ideas` | Generate ideas from clusters |
|
||||||
|
|
||||||
|
### Content Ideas
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/api/v1/planner/ideas/` | `ContentIdeaViewSet.list` | List ideas |
|
||||||
|
| POST | `/api/v1/planner/ideas/` | `ContentIdeaViewSet.create` | Create idea manually |
|
||||||
|
| POST | `/api/v1/planner/ideas/create_tasks/` | `ContentIdeaViewSet.create_tasks` | Convert ideas to tasks |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Business Logic
|
||||||
|
|
||||||
|
### Auto Clustering (AI)
|
||||||
|
|
||||||
|
**Trigger:** User clicks "Auto Cluster" button
|
||||||
|
**AI Function:** `AutoClusterFunction`
|
||||||
|
**Credit Cost:** Per batch (configurable)
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. User selects keywords (or all unclustered)
|
||||||
|
2. Frontend calls `POST /clusters/auto_cluster/`
|
||||||
|
3. Backend validates minimum 5 keywords
|
||||||
|
4. AIEngine executes `AutoClusterFunction`:
|
||||||
|
- Sends keywords to GPT-4
|
||||||
|
- AI groups by semantic similarity
|
||||||
|
- Returns cluster assignments
|
||||||
|
5. Creates/updates `Clusters` records
|
||||||
|
6. Assigns keywords to clusters
|
||||||
|
7. Returns created clusters
|
||||||
|
|
||||||
|
### Generate Ideas (AI)
|
||||||
|
|
||||||
|
**Trigger:** User clicks "Generate Ideas" on cluster(s)
|
||||||
|
**AI Function:** `GenerateIdeasFunction`
|
||||||
|
**Credit Cost:** Per idea generated
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. User selects clusters
|
||||||
|
2. Frontend calls `POST /clusters/generate_ideas/`
|
||||||
|
3. Backend validates clusters have keywords
|
||||||
|
4. AIEngine executes `GenerateIdeasFunction`:
|
||||||
|
- Analyzes cluster keywords
|
||||||
|
- Generates content titles + briefs
|
||||||
|
- Suggests word counts and content types
|
||||||
|
5. Creates `ContentIdeas` records
|
||||||
|
6. Returns created ideas
|
||||||
|
|
||||||
|
### Add Keywords to Workflow
|
||||||
|
|
||||||
|
**Trigger:** User selects SeedKeywords in setup
|
||||||
|
**Credit Cost:** None
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. User browses global SeedKeywords
|
||||||
|
2. Selects keywords to add to their sector
|
||||||
|
3. Frontend calls `POST /keywords/add_to_workflow/`
|
||||||
|
4. Backend creates `Keywords` records linked to SeedKeywords
|
||||||
|
5. Keywords appear in Planner for clustering
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Frontend Pages
|
||||||
|
|
||||||
|
### Keywords Page (`/planner/keywords`)
|
||||||
|
|
||||||
|
- Table of all keywords with filtering
|
||||||
|
- Bulk actions: delete, update status
|
||||||
|
- Add keywords from SeedKeyword library
|
||||||
|
- Import keywords from CSV
|
||||||
|
- Assign to clusters manually
|
||||||
|
|
||||||
|
### Clusters Page (`/planner/clusters`)
|
||||||
|
|
||||||
|
- Grid/list of clusters
|
||||||
|
- Auto-cluster action (AI)
|
||||||
|
- Generate ideas action (AI)
|
||||||
|
- View keywords in cluster
|
||||||
|
- Cluster status management
|
||||||
|
|
||||||
|
### Ideas Page (`/planner/ideas`)
|
||||||
|
|
||||||
|
- Table of content ideas
|
||||||
|
- Convert to tasks action
|
||||||
|
- Edit idea details
|
||||||
|
- Priority management
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Integration Points
|
||||||
|
|
||||||
|
| From | To | Trigger |
|
||||||
|
|------|----|---------|
|
||||||
|
| SeedKeywords | Keywords | Add to workflow |
|
||||||
|
| Keywords | Clusters | Auto clustering |
|
||||||
|
| Clusters | ContentIdeas | Generate ideas |
|
||||||
|
| ContentIdeas | Tasks | Create tasks |
|
||||||
|
| Automation Stage 1 | Clusters | Automated clustering |
|
||||||
|
| Automation Stage 2 | ContentIdeas | Automated idea generation |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
| Issue | Cause | Fix |
|
||||||
|
|-------|-------|-----|
|
||||||
|
| Clustering fails | Less than 5 keywords | Ensure minimum keywords |
|
||||||
|
| Ideas not generating | Cluster has no keywords | Assign keywords to cluster |
|
||||||
|
| Keywords not showing | Wrong site/sector filter | Check active site/sector |
|
||||||
|
| Duplicate keywords | Same keyword added twice | Check seed_keyword reference |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
|
||||||
|
| Feature | Status | Description |
|
||||||
|
|---------|--------|-------------|
|
||||||
|
| Keyword import improvements | 🔜 Planned | Better CSV parsing and validation |
|
||||||
|
| Cluster merging | 🔜 Planned | Merge similar clusters |
|
||||||
|
| Idea prioritization | 🔜 Planned | AI-based priority scoring |
|
||||||
184
docs/10-MODULES/PUBLISHER.md
Normal file
184
docs/10-MODULES/PUBLISHER.md
Normal file
@@ -0,0 +1,184 @@
|
|||||||
|
# Publisher Module
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Status:** ✅ Active
|
||||||
|
**Backend Path:** `backend/igny8_core/modules/publisher/` + `backend/igny8_core/business/publishing/`
|
||||||
|
**Frontend Path:** N/A (API-only module)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
|
||||||
|
| What | File | Key Items |
|
||||||
|
|------|------|-----------|
|
||||||
|
| Views | `modules/publisher/views.py` | `PublishingRecordViewSet`, `DeploymentViewSet`, `PublishContentViewSet` |
|
||||||
|
| Models | `business/publishing/models.py` | `PublishingRecord`, `DeploymentRecord` |
|
||||||
|
| Services | `business/publishing/services/*.py` | Publishing orchestration |
|
||||||
|
| URLs | `modules/publisher/urls.py` | Publisher endpoints |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
The Publisher module manages:
|
||||||
|
- Content publishing pipeline
|
||||||
|
- Publishing record tracking
|
||||||
|
- Deployment management
|
||||||
|
- Multi-destination publishing
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Data Models
|
||||||
|
|
||||||
|
### PublishingRecord
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| site | FK | Parent site |
|
||||||
|
| content | FK | Source content |
|
||||||
|
| destination | CharField | wordpress/ghost/webflow |
|
||||||
|
| external_id | CharField | ID on destination platform |
|
||||||
|
| external_url | URLField | Published URL |
|
||||||
|
| status | CharField | pending/published/failed/retracted |
|
||||||
|
| published_at | DateTime | Publication time |
|
||||||
|
| metadata | JSON | Additional data |
|
||||||
|
| created_at | DateTime | Record creation |
|
||||||
|
|
||||||
|
### DeploymentRecord
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| site | FK | Target site |
|
||||||
|
| deployment_type | CharField | full/incremental |
|
||||||
|
| status | CharField | pending/deploying/completed/failed |
|
||||||
|
| items_deployed | Integer | Number of items |
|
||||||
|
| started_at | DateTime | Start time |
|
||||||
|
| completed_at | DateTime | Completion time |
|
||||||
|
| error_log | TextField | Errors encountered |
|
||||||
|
| metadata | JSON | Deployment details |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/api/v1/publisher/records/` | `PublishingRecordViewSet.list` | List publishing records |
|
||||||
|
| POST | `/api/v1/publisher/records/` | `PublishingRecordViewSet.create` | Create record |
|
||||||
|
| GET | `/api/v1/publisher/deployments/` | `DeploymentViewSet.list` | List deployments |
|
||||||
|
| POST | `/api/v1/publisher/publish/` | `PublishContentViewSet.publish` | Publish content |
|
||||||
|
| GET | `/api/v1/publisher/publish/status/` | `PublishContentViewSet.status` | Get publishing status |
|
||||||
|
| GET | `/api/v1/publisher/site-definition/` | `SiteDefinitionViewSet.list` | Public site definitions |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Publishing Flow
|
||||||
|
|
||||||
|
### Single Content Publish
|
||||||
|
|
||||||
|
**Trigger:** User clicks "Publish" or API call
|
||||||
|
**Flow:**
|
||||||
|
1. Validate content is in publishable state
|
||||||
|
2. Get site integration credentials
|
||||||
|
3. Transform content for destination:
|
||||||
|
- Format HTML for platform
|
||||||
|
- Process images
|
||||||
|
- Map taxonomies
|
||||||
|
4. Push to destination API
|
||||||
|
5. Create `PublishingRecord` with external ID
|
||||||
|
6. Update Content status to `published`
|
||||||
|
|
||||||
|
### Batch Deployment
|
||||||
|
|
||||||
|
**Trigger:** Deployment action
|
||||||
|
**Flow:**
|
||||||
|
1. Create `DeploymentRecord`
|
||||||
|
2. Gather all pending content
|
||||||
|
3. Process each item:
|
||||||
|
- Publish to destination
|
||||||
|
- Create publishing record
|
||||||
|
- Update content status
|
||||||
|
4. Update deployment status
|
||||||
|
5. Log any errors
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Destination Adapters
|
||||||
|
|
||||||
|
### WordPress Adapter
|
||||||
|
|
||||||
|
- Uses WordPress REST API
|
||||||
|
- Supports posts, pages, custom post types
|
||||||
|
- Handles media upload
|
||||||
|
- Maps categories/tags
|
||||||
|
|
||||||
|
### Ghost Adapter (Planned)
|
||||||
|
|
||||||
|
- Ghost Admin API integration
|
||||||
|
- Post publishing
|
||||||
|
- Image handling
|
||||||
|
|
||||||
|
### Webflow Adapter (Planned)
|
||||||
|
|
||||||
|
- Webflow CMS API
|
||||||
|
- Collection item publishing
|
||||||
|
- Asset management
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Publishing States
|
||||||
|
|
||||||
|
| Status | Description |
|
||||||
|
|--------|-------------|
|
||||||
|
| pending | Ready to publish |
|
||||||
|
| published | Successfully published |
|
||||||
|
| failed | Publishing failed |
|
||||||
|
| retracted | Unpublished/removed |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Site Definition Endpoint
|
||||||
|
|
||||||
|
**Purpose:** Public endpoint for headless CMS use cases
|
||||||
|
|
||||||
|
Returns site structure for external consumption:
|
||||||
|
- Site metadata
|
||||||
|
- Available taxonomies
|
||||||
|
- Content types
|
||||||
|
- URL structure
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Integration Points
|
||||||
|
|
||||||
|
| From | To | Trigger |
|
||||||
|
|------|----|---------|
|
||||||
|
| Writer | Publisher | Publish action |
|
||||||
|
| Automation Stage 7 | Publisher | Auto-publish (future) |
|
||||||
|
| Publisher | Integrations | Destination APIs |
|
||||||
|
| Publisher | Content | Status updates |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
| Issue | Cause | Fix |
|
||||||
|
|-------|-------|-----|
|
||||||
|
| Publish failed | Invalid credentials | Check integration settings |
|
||||||
|
| Duplicate posts | Published twice | Check existing publishing record |
|
||||||
|
| Images missing | Upload failed | Check media library access |
|
||||||
|
| Wrong category | Mapping issue | Verify taxonomy sync |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
|
||||||
|
| Feature | Status | Description |
|
||||||
|
|---------|--------|-------------|
|
||||||
|
| Ghost integration | 🔜 Planned | Ghost CMS publishing |
|
||||||
|
| Webflow integration | 🔜 Planned | Webflow publishing |
|
||||||
|
| Scheduled publishing | 🔜 Planned | Future-date publishing |
|
||||||
|
| Republish detection | 🔜 Planned | Detect and handle updates |
|
||||||
|
| Publishing queue | 🔜 Planned | Batch publishing with queue |
|
||||||
286
docs/10-MODULES/SYSTEM-SETTINGS.md
Normal file
286
docs/10-MODULES/SYSTEM-SETTINGS.md
Normal file
@@ -0,0 +1,286 @@
|
|||||||
|
# System Settings Module
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Status:** ✅ Active
|
||||||
|
**Backend Path:** `backend/igny8_core/modules/system/`
|
||||||
|
**Frontend Path:** `frontend/src/pages/Settings/`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
|
||||||
|
| What | File | Key Items |
|
||||||
|
|------|------|-----------|
|
||||||
|
| Global Models | `modules/system/global_settings_models.py` | `GlobalIntegrationSettings`, `GlobalAIPrompt`, `GlobalAuthorProfile` |
|
||||||
|
| Account Models | `modules/system/settings_models.py` | `ModuleEnableSettings`, `IntegrationSettings`, `AISettings` |
|
||||||
|
| Views | `modules/system/settings_views.py` | Settings ViewSets |
|
||||||
|
| Integration Views | `modules/system/integration_views.py` | AI integration settings |
|
||||||
|
| Frontend | `pages/Settings/*.tsx` | Settings pages |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
The System Settings module manages:
|
||||||
|
- Platform-wide global settings (API keys, defaults)
|
||||||
|
- Per-account settings overrides
|
||||||
|
- AI prompts and configurations
|
||||||
|
- Module enable/disable
|
||||||
|
- Author profiles and content strategies
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Settings Hierarchy
|
||||||
|
|
||||||
|
```
|
||||||
|
Global Settings (Platform-wide)
|
||||||
|
↓
|
||||||
|
Account Settings (Per-account overrides)
|
||||||
|
↓
|
||||||
|
User Settings (Per-user preferences)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** User > Account > Global
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Global Settings Models
|
||||||
|
|
||||||
|
### GlobalIntegrationSettings (Singleton)
|
||||||
|
|
||||||
|
**Admin:** `/admin/system/globalintegrationsettings/`
|
||||||
|
**Purpose:** Platform-wide API keys and defaults
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| openai_api_key | CharField | OpenAI API key (all accounts) |
|
||||||
|
| openai_model | CharField | Default model (gpt-4o-mini) |
|
||||||
|
| openai_temperature | Float | Default temperature (0.7) |
|
||||||
|
| openai_max_tokens | Integer | Default max tokens (8192) |
|
||||||
|
| dalle_api_key | CharField | DALL-E API key |
|
||||||
|
| dalle_model | CharField | Default model (dall-e-3) |
|
||||||
|
| dalle_size | CharField | Default size (1024x1024) |
|
||||||
|
| dalle_quality | CharField | Default quality (standard) |
|
||||||
|
| dalle_style | CharField | Default style (vivid) |
|
||||||
|
| anthropic_api_key | CharField | Anthropic API key |
|
||||||
|
| runware_api_key | CharField | Runware API key |
|
||||||
|
|
||||||
|
**Critical:** This is a singleton (only 1 record, pk=1). All accounts use these API keys.
|
||||||
|
|
||||||
|
### GlobalAIPrompt
|
||||||
|
|
||||||
|
**Admin:** `/admin/system/globalaiprompt/`
|
||||||
|
**Purpose:** Default AI prompt templates
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| prompt_type | CharField | clustering/ideas/content_generation/etc. |
|
||||||
|
| prompt_value | TextField | The actual prompt text |
|
||||||
|
| description | TextField | What this prompt does |
|
||||||
|
| variables | JSON | Available variables ({keyword}, {industry}) |
|
||||||
|
| version | Integer | Prompt version |
|
||||||
|
| is_active | Boolean | Enable/disable |
|
||||||
|
|
||||||
|
### GlobalAuthorProfile
|
||||||
|
|
||||||
|
**Admin:** `/admin/system/globalauthorprofile/`
|
||||||
|
**Purpose:** Default author persona templates
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| name | CharField | Profile name |
|
||||||
|
| description | TextField | Description |
|
||||||
|
| tone | CharField | professional/casual/technical |
|
||||||
|
| language | CharField | en/es/fr |
|
||||||
|
| structure_template | JSON | Content structure config |
|
||||||
|
| category | CharField | saas/ecommerce/blog/technical |
|
||||||
|
| is_active | Boolean | Enable/disable |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Account Settings Models
|
||||||
|
|
||||||
|
### IntegrationSettings
|
||||||
|
|
||||||
|
**Purpose:** Per-account AI model overrides (NOT API keys)
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| integration_type | CharField | openai/runware/image_generation |
|
||||||
|
| config | JSON | Model, temperature, max_tokens overrides |
|
||||||
|
| is_active | Boolean | Enable/disable |
|
||||||
|
|
||||||
|
**Important:**
|
||||||
|
- Free plan cannot create overrides
|
||||||
|
- Starter/Growth/Scale can override model/settings
|
||||||
|
- API keys ALWAYS come from GlobalIntegrationSettings
|
||||||
|
|
||||||
|
### ModuleEnableSettings
|
||||||
|
|
||||||
|
**Purpose:** Enable/disable modules per account
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| planner_enabled | Boolean | Enable Planner |
|
||||||
|
| writer_enabled | Boolean | Enable Writer |
|
||||||
|
| thinker_enabled | Boolean | Enable Thinker (AI settings) |
|
||||||
|
| automation_enabled | Boolean | Enable Automation |
|
||||||
|
| site_builder_enabled | Boolean | Enable Site Builder |
|
||||||
|
| linker_enabled | Boolean | Enable Linker |
|
||||||
|
| optimizer_enabled | Boolean | Enable Optimizer |
|
||||||
|
| publisher_enabled | Boolean | Enable Publisher |
|
||||||
|
|
||||||
|
**Current Implementation:**
|
||||||
|
- Controls sidebar navigation visibility
|
||||||
|
- ⚠️ **Pending:** Extend to other pages and references
|
||||||
|
|
||||||
|
### AIPrompt (Account-Level)
|
||||||
|
|
||||||
|
**Purpose:** Per-account prompt customizations
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| prompt_type | CharField | Prompt type |
|
||||||
|
| prompt_value | TextField | Current prompt (custom or default) |
|
||||||
|
| default_prompt | TextField | Original default (for reset) |
|
||||||
|
| is_customized | Boolean | True if user modified |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
|
||||||
|
### Integration Settings
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/api/v1/system/settings/integrations/openai/` | Get OpenAI settings | Get current model/params |
|
||||||
|
| PUT | `/api/v1/system/settings/integrations/openai/` | Save OpenAI settings | Save overrides |
|
||||||
|
| GET | `/api/v1/system/settings/integrations/image_generation/` | Get image settings | Get DALL-E/Runware settings |
|
||||||
|
| PUT | `/api/v1/system/settings/integrations/image_generation/` | Save image settings | Save overrides |
|
||||||
|
| POST | `/api/v1/system/settings/integrations/test/` | Test connection | Test API connectivity |
|
||||||
|
|
||||||
|
### Prompts
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/api/v1/system/prompts/` | List prompts | Get all prompts |
|
||||||
|
| GET | `/api/v1/system/prompts/{type}/` | Get prompt | Get specific prompt |
|
||||||
|
| PUT | `/api/v1/system/prompts/{type}/` | Save prompt | Save customization |
|
||||||
|
| POST | `/api/v1/system/prompts/{type}/reset/` | Reset prompt | Reset to default |
|
||||||
|
|
||||||
|
### Module Settings
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/api/v1/system/modules/` | Get module settings | Get enable/disable state |
|
||||||
|
| PUT | `/api/v1/system/modules/` | Save module settings | Update enabled modules |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Settings Flow (Frontend → Backend)
|
||||||
|
|
||||||
|
### Getting OpenAI Settings
|
||||||
|
|
||||||
|
1. Frontend requests: `GET /system/settings/integrations/openai/`
|
||||||
|
2. Backend checks account's `IntegrationSettings`
|
||||||
|
3. Gets global defaults from `GlobalIntegrationSettings`
|
||||||
|
4. Merges: account overrides > global defaults
|
||||||
|
5. Returns (NEVER includes API keys):
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"model": "gpt-4o-mini",
|
||||||
|
"temperature": 0.7,
|
||||||
|
"max_tokens": 8192,
|
||||||
|
"using_global": true
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Saving OpenAI Settings
|
||||||
|
|
||||||
|
1. Frontend sends: `PUT /system/settings/integrations/openai/`
|
||||||
|
2. Backend STRIPS any API key fields (security)
|
||||||
|
3. Validates account plan allows overrides
|
||||||
|
4. Saves to `IntegrationSettings.config`
|
||||||
|
5. Returns updated settings
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Module Enable/Disable
|
||||||
|
|
||||||
|
### How It Works (Current)
|
||||||
|
|
||||||
|
1. On app load, frontend fetches module settings
|
||||||
|
2. `useModuleStore.isModuleEnabled(name)` checks state
|
||||||
|
3. `AppSidebar.tsx` conditionally renders menu items:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
if (isModuleEnabled('linker')) {
|
||||||
|
workflowItems.push({
|
||||||
|
name: "Linker",
|
||||||
|
path: "/linker/content",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Current Limitations (⚠️ Pending Implementation)
|
||||||
|
|
||||||
|
- Only hides sidebar menu items
|
||||||
|
- Direct URL access still works
|
||||||
|
- Other page references still show module links
|
||||||
|
- Dashboard cards may still show disabled modules
|
||||||
|
|
||||||
|
### Required Extension
|
||||||
|
|
||||||
|
Need to add `ModuleGuard` component to:
|
||||||
|
- Route-level protection
|
||||||
|
- Dashboard cards/widgets
|
||||||
|
- Cross-module references
|
||||||
|
- Settings page links
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Frontend Pages
|
||||||
|
|
||||||
|
### AI Settings (`/settings/ai`)
|
||||||
|
|
||||||
|
- OpenAI model selection
|
||||||
|
- Temperature and max tokens
|
||||||
|
- Image generation settings
|
||||||
|
- Test connection button
|
||||||
|
|
||||||
|
### Prompts (`/thinker/prompts`)
|
||||||
|
|
||||||
|
- List all prompt types
|
||||||
|
- Edit prompt text
|
||||||
|
- Reset to default
|
||||||
|
- Variable reference
|
||||||
|
|
||||||
|
### Module Settings (Admin Only)
|
||||||
|
|
||||||
|
- Enable/disable modules
|
||||||
|
- Per-account configuration
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
| Issue | Cause | Fix |
|
||||||
|
|-------|-------|-----|
|
||||||
|
| Settings not saving | Plan restriction | Upgrade plan |
|
||||||
|
| API key exposed | Security flaw | Should never happen - check code |
|
||||||
|
| Module still visible | Cache stale | Clear cache, reload |
|
||||||
|
| Prompt reset not working | default_prompt missing | Re-run migration |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
|
||||||
|
| Feature | Status | Description |
|
||||||
|
|---------|--------|-------------|
|
||||||
|
| Module guard extension | 🔜 Pending | Extend disable to all pages, not just sidebar |
|
||||||
|
| AIModelConfig database | 🔜 Planned | Move model pricing to database |
|
||||||
|
| Strategy templates | 🔜 Planned | Global strategy library |
|
||||||
|
| Per-user preferences | 🔜 Planned | User-level setting overrides |
|
||||||
293
docs/10-MODULES/WRITER.md
Normal file
293
docs/10-MODULES/WRITER.md
Normal file
@@ -0,0 +1,293 @@
|
|||||||
|
# Writer Module
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Status:** ✅ Active
|
||||||
|
**Backend Path:** `backend/igny8_core/modules/writer/`
|
||||||
|
**Frontend Path:** `frontend/src/pages/Writer/`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Reference
|
||||||
|
|
||||||
|
| What | File | Key Items |
|
||||||
|
|------|------|-----------|
|
||||||
|
| Models | `modules/writer/models.py` | `Tasks`, `Content`, `Images`, `ContentTaxonomy` |
|
||||||
|
| Views | `modules/writer/views.py` | `TaskViewSet`, `ContentViewSet`, `ImageViewSet` |
|
||||||
|
| AI Functions | `ai/functions/content.py` | `GenerateContentFunction` |
|
||||||
|
| AI Functions | `ai/functions/images.py` | `GenerateImagesFunction`, `GenerateImagePromptsFunction` |
|
||||||
|
| Frontend Pages | `pages/Writer/Tasks.tsx` | Task management |
|
||||||
|
| Frontend Pages | `pages/Writer/Content.tsx` | Content listing |
|
||||||
|
| Frontend Pages | `pages/Writer/ContentViewer.tsx` | Content preview/edit |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
The Writer module manages the content creation pipeline:
|
||||||
|
|
||||||
|
```
|
||||||
|
ContentIdeas → Tasks → Content → Images → Review → Publish
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Data Models
|
||||||
|
|
||||||
|
### Tasks
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| site | FK | Parent site |
|
||||||
|
| sector | FK | Parent sector |
|
||||||
|
| content_idea | FK | Source idea (nullable) |
|
||||||
|
| title | CharField | Task/content title |
|
||||||
|
| description | TextField | Task brief |
|
||||||
|
| target_keywords | JSON | Keywords to target |
|
||||||
|
| content_type | CharField | blog_post/guide/comparison/etc. |
|
||||||
|
| word_count_target | Integer | Target word count |
|
||||||
|
| status | CharField | pending/in_progress/completed/cancelled |
|
||||||
|
| assigned_to | FK | Assigned user (nullable) |
|
||||||
|
| due_date | DateTime | Due date (nullable) |
|
||||||
|
| priority | Integer | Priority level |
|
||||||
|
| created_at | DateTime | Creation date |
|
||||||
|
|
||||||
|
### Content
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| site | FK | Parent site |
|
||||||
|
| sector | FK | Parent sector |
|
||||||
|
| task | FK | Source task (nullable) |
|
||||||
|
| title | CharField | Content title |
|
||||||
|
| slug | SlugField | URL slug |
|
||||||
|
| content_body | TextField | HTML content |
|
||||||
|
| excerpt | TextField | Short summary |
|
||||||
|
| meta_title | CharField | SEO meta title |
|
||||||
|
| meta_description | CharField | SEO meta description |
|
||||||
|
| word_count | Integer | Actual word count |
|
||||||
|
| status | CharField | draft/review/approved/published |
|
||||||
|
| content_type | CharField | post/page/product |
|
||||||
|
| content_structure | CharField | article/guide/comparison/review/listicle |
|
||||||
|
| source | CharField | igny8/wordpress/manual |
|
||||||
|
| wordpress_id | Integer | WP post ID (if synced) |
|
||||||
|
| published_at | DateTime | Publication date |
|
||||||
|
| created_at | DateTime | Creation date |
|
||||||
|
|
||||||
|
### Images
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| site | FK | Parent site |
|
||||||
|
| sector | FK | Parent sector |
|
||||||
|
| content | FK | Parent content |
|
||||||
|
| image_type | CharField | featured/desktop/mobile/in_article |
|
||||||
|
| prompt | TextField | Generation prompt |
|
||||||
|
| image_url | URLField | Image URL |
|
||||||
|
| alt_text | CharField | Alt text for SEO |
|
||||||
|
| status | CharField | pending/generating/completed/failed |
|
||||||
|
| position | Integer | Order for in-article images |
|
||||||
|
| provider | CharField | dalle/runware |
|
||||||
|
| model | CharField | dall-e-3/runware:97@1 |
|
||||||
|
| created_at | DateTime | Creation date |
|
||||||
|
|
||||||
|
### ContentTaxonomy
|
||||||
|
|
||||||
|
| Field | Type | Purpose |
|
||||||
|
|-------|------|---------|
|
||||||
|
| account | FK | Owner account |
|
||||||
|
| site | FK | Parent site |
|
||||||
|
| name | CharField | Category/tag name |
|
||||||
|
| slug | SlugField | URL slug |
|
||||||
|
| taxonomy_type | CharField | category/tag |
|
||||||
|
| parent | FK | Parent taxonomy (for hierarchy) |
|
||||||
|
| description | TextField | Description |
|
||||||
|
| wordpress_id | Integer | WP term ID (if synced) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
|
||||||
|
### Tasks
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/api/v1/writer/tasks/` | `TaskViewSet.list` | List tasks |
|
||||||
|
| POST | `/api/v1/writer/tasks/` | `TaskViewSet.create` | Create task |
|
||||||
|
| POST | `/api/v1/writer/tasks/bulk_create/` | `TaskViewSet.bulk_create` | Create multiple tasks |
|
||||||
|
| POST | `/api/v1/writer/tasks/{id}/generate_content/` | `TaskViewSet.generate_content` | AI content generation |
|
||||||
|
|
||||||
|
### Content
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/api/v1/writer/content/` | `ContentViewSet.list` | List content |
|
||||||
|
| POST | `/api/v1/writer/content/` | `ContentViewSet.create` | Create content manually |
|
||||||
|
| PUT | `/api/v1/writer/content/{id}/` | `ContentViewSet.update` | Update content |
|
||||||
|
| POST | `/api/v1/writer/content/{id}/update_content/` | `ContentViewSet.update_content` | Update with validation |
|
||||||
|
| POST | `/api/v1/writer/content/{id}/generate_images/` | `ContentViewSet.generate_images` | Generate images |
|
||||||
|
| POST | `/api/v1/writer/content/{id}/publish_to_wordpress/` | `ContentViewSet.publish_to_wordpress` | Publish to WP |
|
||||||
|
|
||||||
|
### Images
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/api/v1/writer/images/` | `ImageViewSet.list` | List images |
|
||||||
|
| POST | `/api/v1/writer/images/generate_for_content/` | `ImageViewSet.generate_for_content` | Generate images |
|
||||||
|
| POST | `/api/v1/writer/images/regenerate/` | `ImageViewSet.regenerate` | Regenerate image |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Business Logic
|
||||||
|
|
||||||
|
### Content Generation (AI)
|
||||||
|
|
||||||
|
**Trigger:** User clicks "Generate" on task
|
||||||
|
**AI Function:** `GenerateContentFunction`
|
||||||
|
**Credit Cost:** Per 100 words generated
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. User has task with title, keywords, word count target
|
||||||
|
2. Frontend calls `POST /tasks/{id}/generate_content/`
|
||||||
|
3. Backend validates task and credits
|
||||||
|
4. AIEngine executes `GenerateContentFunction`:
|
||||||
|
- Loads account's AI prompts and strategy
|
||||||
|
- Sends to GPT-4 with structured output
|
||||||
|
- Receives HTML content with proper structure
|
||||||
|
5. Creates `Content` record linked to task
|
||||||
|
6. Updates task status to `completed`
|
||||||
|
7. Returns content for review
|
||||||
|
|
||||||
|
### Image Prompt Generation (AI)
|
||||||
|
|
||||||
|
**Trigger:** Part of content generation or explicit
|
||||||
|
**AI Function:** `GenerateImagePromptsFunction`
|
||||||
|
**Credit Cost:** Per prompt
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. Content exists with body
|
||||||
|
2. AI analyzes content sections
|
||||||
|
3. Generates prompts for:
|
||||||
|
- Featured image (1)
|
||||||
|
- In-article images (configurable count)
|
||||||
|
4. Creates `Images` records with prompts, status=pending
|
||||||
|
|
||||||
|
### Image Generation (AI)
|
||||||
|
|
||||||
|
**Trigger:** User clicks "Generate Images" or automation
|
||||||
|
**AI Function:** `GenerateImagesFunction`
|
||||||
|
**Credit Cost:** Per image
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. Image record exists with prompt, status=pending
|
||||||
|
2. Backend calls DALL-E or Runware API
|
||||||
|
3. Receives image URL
|
||||||
|
4. Updates `Images` record with URL, status=completed
|
||||||
|
|
||||||
|
### WordPress Publishing
|
||||||
|
|
||||||
|
**Trigger:** User clicks "Publish to WordPress"
|
||||||
|
**Credit Cost:** None
|
||||||
|
|
||||||
|
**Flow:**
|
||||||
|
1. Content is in `approved` status
|
||||||
|
2. Site has WordPress integration configured
|
||||||
|
3. Backend calls WordPress REST API:
|
||||||
|
- Creates/updates post
|
||||||
|
- Uploads featured image
|
||||||
|
- Sets categories/tags
|
||||||
|
4. Updates Content with `wordpress_id`
|
||||||
|
5. Sets status to `published`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Content Structures
|
||||||
|
|
||||||
|
| Structure | Purpose | Typical Sections |
|
||||||
|
|-----------|---------|------------------|
|
||||||
|
| article | General blog post | Intro, Body, Conclusion |
|
||||||
|
| guide | How-to content | Steps, Tips, Summary |
|
||||||
|
| comparison | Product comparison | Features, Pros/Cons, Verdict |
|
||||||
|
| review | Product review | Overview, Features, Rating |
|
||||||
|
| listicle | List-based content | Numbered items with details |
|
||||||
|
| pillar | Long-form authority | Multiple sections with depth |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Frontend Pages
|
||||||
|
|
||||||
|
### Tasks Page (`/writer/tasks`)
|
||||||
|
|
||||||
|
- Table of all tasks
|
||||||
|
- Filter by status, assigned user
|
||||||
|
- Generate content action
|
||||||
|
- Bulk actions
|
||||||
|
|
||||||
|
### Content Page (`/writer/content`)
|
||||||
|
|
||||||
|
- Table of all content
|
||||||
|
- Filter by status, content type
|
||||||
|
- Quick preview
|
||||||
|
- Publish actions
|
||||||
|
|
||||||
|
### Content Viewer (`/writer/content/{id}`)
|
||||||
|
|
||||||
|
- Full content preview
|
||||||
|
- Edit mode
|
||||||
|
- Image management
|
||||||
|
- Publish to WordPress
|
||||||
|
- Status management
|
||||||
|
|
||||||
|
### Draft/Review/Published Tabs
|
||||||
|
|
||||||
|
- Filtered views by status
|
||||||
|
- Different actions per status
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Content Status Flow
|
||||||
|
|
||||||
|
```
|
||||||
|
draft → review → approved → published
|
||||||
|
↓
|
||||||
|
(rejected) → draft (revision)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Integration Points
|
||||||
|
|
||||||
|
| From | To | Trigger |
|
||||||
|
|------|----|---------|
|
||||||
|
| ContentIdeas | Tasks | Create tasks |
|
||||||
|
| Tasks | Content | Generate content |
|
||||||
|
| Content | Images | Generate images |
|
||||||
|
| Content | WordPress | Publish |
|
||||||
|
| WordPress | Content | Sync imports |
|
||||||
|
| Automation Stage 4 | Content | Automated generation |
|
||||||
|
| Automation Stage 5-6 | Images | Automated image generation |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Issues
|
||||||
|
|
||||||
|
| Issue | Cause | Fix |
|
||||||
|
|-------|-------|-----|
|
||||||
|
| Content too short | Low word count target | Increase target in task |
|
||||||
|
| Images not generating | No prompts created | Run image prompt generation first |
|
||||||
|
| WordPress publish fails | Invalid credentials | Check integration settings |
|
||||||
|
| Content stuck in draft | No manual status update | Update status via UI or API |
|
||||||
|
| Duplicate content | Re-running generation | Check if content already exists |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
|
||||||
|
| Feature | Status | Description |
|
||||||
|
|---------|--------|-------------|
|
||||||
|
| Content revisions | 🔜 Planned | Track content version history |
|
||||||
|
| Multi-language | 🔜 Planned | Generate content in different languages |
|
||||||
|
| Batch generation | 🔜 Planned | Generate multiple content pieces at once |
|
||||||
|
| Regenerate sections | 🔜 Planned | AI regenerate specific sections |
|
||||||
@@ -1,73 +0,0 @@
|
|||||||
# REST API Reference
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Summarize the major API groups, their base paths, and key actions as implemented in routing and viewsets.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Root routing: `backend/igny8_core/urls.py`
|
|
||||||
- Auth endpoints: `backend/igny8_core/auth/urls.py` (APIView + routers)
|
|
||||||
- Account management: `backend/igny8_core/api/urls.py`
|
|
||||||
- Planner: `backend/igny8_core/modules/planner/urls.py`
|
|
||||||
- Writer: `backend/igny8_core/modules/writer/urls.py`
|
|
||||||
- System: `backend/igny8_core/modules/system/urls.py`
|
|
||||||
- Billing: `backend/igny8_core/business/billing/urls.py`, `backend/igny8_core/modules/billing/urls.py`
|
|
||||||
- Automation: `backend/igny8_core/business/automation/urls.py`
|
|
||||||
- Integration: `backend/igny8_core/modules/integration/urls.py`
|
|
||||||
- Linker/Optimizer/Publisher: respective `modules/*/urls.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Provide RESTful CRUD endpoints for core resources (auth, planner, writer, billing, automation, integration, etc.) plus custom actions for domain workflows (automation run, billing payments, integration sync, content generation).
|
|
||||||
|
|
||||||
## Detailed Behavior (by group)
|
|
||||||
- Authentication (`/api/v1/auth/`):
|
|
||||||
- Routers: groups, users, accounts, subscriptions, site-access, plans, sites, sectors, industries, seed-keywords.
|
|
||||||
- APIViews: register, login, change password, refresh token; responses issue JWTs and session login.
|
|
||||||
- CSV admin helpers exposed separately under `/admin/igny8_core_auth/...` for industries/seed keywords.
|
|
||||||
- Account management (`/api/v1/account/`):
|
|
||||||
- Settings (`settings/` get/patch), team (`team/` list/create, `team/<id>/` delete), usage analytics (`usage/analytics/`).
|
|
||||||
- Planner (`/api/v1/planner/`):
|
|
||||||
- `keywords/`, `clusters/`, `ideas/` (CRUD, filtering/search/ordering, bulk delete/update, bulk add from seed).
|
|
||||||
- Writer (`/api/v1/writer/`):
|
|
||||||
- `tasks/`, `images/`, `content/`, `taxonomies/` (CRUD, filtering/search/ordering).
|
|
||||||
- Custom actions: `tasks/auto_generate_content` (AI generation), `tasks/bulk_*`, `images/file`, grouped image utilities; content/taxonomy helpers in viewset.
|
|
||||||
- System (`/api/v1/system/`): system settings, prompts, author profiles, etc. (see module docs).
|
|
||||||
- Billing (`/api/v1/billing/`):
|
|
||||||
- Invoices (`invoices/`, detail, `download_pdf`), payments (`payments/`, `available_methods`, manual payment), credit packages (`credit-packages/`, purchase), transactions (`transactions/`), payment methods (`payment-methods/`, `set_default`), credits (`credits/balance`, `credits/usage`, `credits/transactions`).
|
|
||||||
- Admin billing under `/api/v1/admin/` (see module billing admin URLs).
|
|
||||||
- Automation (`/api/v1/automation/`):
|
|
||||||
- Config (`config`, `update_config`), run (`run_now`), current run, history, logs, estimate, pipeline_overview, current_processing, pause/resume/cancel.
|
|
||||||
- Integration (`/api/v1/integration/`):
|
|
||||||
- `integrations/` CRUD, `test_connection`, collection-level `test-connection`, `sync`, `sync_status`, `update-structure`, `get_content_types`, `debug_status`, `sync_wordpress_content`.
|
|
||||||
- Webhooks: `/webhooks/wordpress/status/`, `/webhooks/wordpress/metadata/`.
|
|
||||||
- Linker/Optimizer/Publisher: endpoints under `/api/v1/linker/`, `/api/v1/optimizer/`, `/api/v1/publisher/` (viewsets for respective domain resources; see module docs).
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Auth/account models, planner models (Keywords/Clusters/ContentIdeas), writer models (Tasks/Content/Images/Taxonomy), billing models (invoices/payments/credits), automation models (runs/configs), integration models (SiteIntegration/SyncEvent), and other module models as routed above.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Requests hit module routers; viewsets/actions enforce permissions, throttles, and tenancy; serializers map models; responses use unified format.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Auth JWTs used by all modules.
|
|
||||||
- Billing credits checked in writer/automation flows; billing endpoints expose balances/usage.
|
|
||||||
- Integration and publishing rely on writer content; automation depends on planner/writer data.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Managed by respective viewsets (e.g., task status, automation status, invoice/payment status).
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Unified error responses; 4xx/5xx codes per viewset logic; throttling per scope.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Base viewsets enforce account/site/sector filtering; API key auth supports WordPress paths; admin/developer/system roles may bypass filters as coded in base viewsets/auth middleware.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- Credits enforced in writer/automation service calls; billing endpoints expose ledger/usage/balance; insufficient credits return 402 where applicable.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- Automation actions enqueue Celery runs; writer content generation may enqueue AI tasks; other modules mostly synchronous.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Consistent `/api/v1/` namespace with per-module routers.
|
|
||||||
- OpenAPI schema available at `/api/schema`, Swagger at `/api/docs`, Redoc at `/api/redoc`.
|
|
||||||
- Use module docs for endpoint-level details; this file outlines the map of API groups and actions.
|
|
||||||
@@ -1,62 +0,0 @@
|
|||||||
# Authentication Endpoints
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Detail authentication-related endpoints for registration, login, password change, token refresh, and auth-adjacent resources (users/accounts/plans/sites/sectors/industries/seed keywords).
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Routing: `backend/igny8_core/auth/urls.py`
|
|
||||||
- Views: `backend/igny8_core/auth/views.py`
|
|
||||||
- Serializers: `backend/igny8_core/auth/serializers.py`
|
|
||||||
- Auth utils: `backend/igny8_core/auth/utils.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Issue JWT access/refresh tokens and session auth on login.
|
|
||||||
- Register users, change passwords, refresh tokens.
|
|
||||||
- Provide CRUD for users, accounts, subscriptions, site access, plans, sites, sectors, industries, seed keywords via routers.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- APIViews:
|
|
||||||
- `POST /api/v1/auth/register/` → `RegisterView`: validates via `RegisterSerializer`, creates user, returns user data.
|
|
||||||
- `POST /api/v1/auth/login/` → `LoginView`: validates credentials, logs in user (session), generates access/refresh JWTs with expiries, returns user data and tokens; 401 on invalid credentials.
|
|
||||||
- `POST /api/v1/auth/change-password/` → `ChangePasswordView`: requires auth, validates old/new passwords, updates password; 400 on invalid current password or validation errors.
|
|
||||||
- `POST /api/v1/auth/refresh/` → `RefreshTokenView`: accepts refresh token, validates, issues new access token with expiry; 401/400 on invalid/expired tokens.
|
|
||||||
- Routers under `/api/v1/auth/`:
|
|
||||||
- `groups/`, `users/`, `accounts/`, `subscriptions/`, `site-access/`, `plans/`, `sites/`, `sectors/`, `industries/`, `seed-keywords/` mapped to corresponding viewsets in `auth/views.py`. These use base viewsets with tenant filtering and role checks (see module docs).
|
|
||||||
- CSV admin helpers (admin UI, not public API): CSV import/export for industry/industrysector/seedkeyword under `/admin/igny8_core_auth/...`.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `User`, `Account`, `Plan`, `Site`, `Sector`, `Industry`, `SeedKeyword`, `Subscription`, `SiteUserAccess`, `Group`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- APIView endpoints use serializers to validate, then issue tokens via auth utils (JWT encode with user_id/account_id, type, exp).
|
|
||||||
- Router endpoints inherit base viewsets; AccountContextMiddleware sets `request.account`; JWT/Session auth applied per settings; permissions enforced per viewset.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Tokens issued here are required by planner/writer/billing/automation/etc.
|
|
||||||
- Account/plan/site data informs tenancy and plan enforcement in middleware and billing limits.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- User creation, password changes, session login, token issuance/refresh.
|
|
||||||
- CRUD on accounts/sites/sectors/industries/seed keywords via routers.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Unified error responses via helpers; login returns 401 on invalid credentials; refresh returns 401/400 on invalid/expired tokens; validation errors return 400 with field errors.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Login loads user with account+plan; middleware later enforces active plan. Router viewsets inherit account/site/sector filtering where applicable.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- None directly; plan data carried on Account; credits handled elsewhere.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- None.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- JWT payload includes user_id, account_id, type, exp/iat; uses `JWT_SECRET_KEY`/`JWT_ALGORITHM` from settings.
|
|
||||||
- Session login occurs alongside JWT issuance to support browser-based flows.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Use provided APIViews for auth; do not handcraft JWTs—use `generate_access_token`/`generate_refresh_token`.
|
|
||||||
- When extending auth resources (e.g., new user fields), update serializers and viewsets; keep router paths stable.
|
|
||||||
- Maintain unified responses and permission/tenancy checks in viewsets.
|
|
||||||
|
|
||||||
@@ -1,65 +0,0 @@
|
|||||||
# Automation Endpoints
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe automation API endpoints that configure, trigger, monitor, and control automation runs.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Routing: `backend/igny8_core/business/automation/urls.py`
|
|
||||||
- Views: `backend/igny8_core/business/automation/views.py`
|
|
||||||
- Services/Tasks: `backend/igny8_core/business/automation/services/automation_service.py`, `automation_logger.py`, `backend/igny8_core/business/automation/tasks.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Expose configuration CRUD (get/update) and operational controls (run, pause, resume, cancel).
|
|
||||||
- Provide run history, status, logs, pipeline overview, credit estimate, and current processing state.
|
|
||||||
- Enforce site/account scoping and authentication.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Base path: `/api/v1/automation/`
|
|
||||||
- Key endpoints (actions on `AutomationViewSet`):
|
|
||||||
- `GET config?site_id=` → get/create `AutomationConfig` for site (includes batch sizes, delays, schedule, flags, last/next run).
|
|
||||||
- `PUT update_config?site_id=` → update config fields (enable, frequency, scheduled_time, batch sizes, delays).
|
|
||||||
- `POST run_now?site_id=` → start a run (checks locks/credits) and enqueue Celery `run_automation_task`.
|
|
||||||
- `GET current_run?site_id=` → current running/paused run details with per-stage results and totals.
|
|
||||||
- `GET history?site_id=` → last 20 runs with status and timestamps.
|
|
||||||
- `GET logs?run_id=&lines=` → tail of automation log file for run.
|
|
||||||
- `GET estimate?site_id=` → estimated credits required plus current balance and sufficiency flag (1.2× buffer).
|
|
||||||
- `GET pipeline_overview?site_id=` → counts/pending by stage across planner/writer models.
|
|
||||||
- `GET current_processing?site_id=&run_id=` → live processing state for active run.
|
|
||||||
- `POST pause?run_id=` / `POST resume?run_id=` / `POST cancel?run_id=` → control run state (pause/resume/cancel).
|
|
||||||
- Permissions/tenancy: `IsAuthenticated`; site fetched with `account=request.user.account`; automation lock is per site; `request.account` set by middleware.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `AutomationConfig`, `AutomationRun`, planner models (Keywords/Clusters/ContentIdeas), writer models (Tasks/Content/Images).
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- View actions call `AutomationService` to start/control runs; long-running work executed by Celery tasks (`run_automation_task`, `resume_automation_task`).
|
|
||||||
- Config updates persist to DB; run state and logs updated as stages progress.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Uses planner/writer data for counts and processing; AI functions run inside service; billing credits checked via account balance before start and inferred from AI task logs during execution.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Run status: running ↔ paused → completed/failed/cancelled; `current_stage` advances per stage; per-stage results stored in `AutomationRun`.
|
|
||||||
- Config `last_run_at`/`next_run_at` updated on scheduled runs.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Missing site_id/run_id → 400; run not found → 404; insufficient credits or concurrent run → 400; server errors → 500.
|
|
||||||
- Logs endpoint 404s when run missing.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All operations scoped to the authenticated user’s account/site; no cross-tenant access; locks are per site.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- Start requires credits ≥ estimated * 1.2; credits consumed by AI tasks are recorded via AI task logs and aggregated per stage.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- Celery tasks for scheduled runs (`check_scheduled_automations`) and pipeline execution/resume.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Cooperative pause/resume/cancel with persisted partial results.
|
|
||||||
- Log access via filesystem tail; structured per-stage results stored in DB for quick status reads.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Trigger runs via `run_now`; avoid bypassing locks/credit checks.
|
|
||||||
- Extend actions by reusing `AutomationService` and Celery tasks; keep site scoping and error handling consistent.
|
|
||||||
|
|
||||||
@@ -1,77 +0,0 @@
|
|||||||
# Billing Endpoints
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Detail billing API endpoints for invoices, payments, credit packages, transactions, payment methods, and credit balance/usage.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Routing: `backend/igny8_core/business/billing/urls.py`, `backend/igny8_core/modules/billing/urls.py`
|
|
||||||
- Views: `backend/igny8_core/business/billing/views.py`, `backend/igny8_core/modules/billing/views.py`
|
|
||||||
- Services: `backend/igny8_core/business/billing/services/credit_service.py`, `invoice_service.py`, `payment_service.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Expose billing resources (invoices, payments, credit packages, payment methods, credit transactions).
|
|
||||||
- Provide credit balance/usage/limits endpoints.
|
|
||||||
- Support manual payment submission and available payment methods lookup.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Base paths:
|
|
||||||
- `/api/v1/billing/` (business billing views via router)
|
|
||||||
- `/api/v1/admin/` (admin billing URLs)
|
|
||||||
- `/api/v1/billing/credits/...` (balance/usage/transactions aliases)
|
|
||||||
- Invoice endpoints (`InvoiceViewSet`):
|
|
||||||
- `GET /invoices/` list account invoices (status filter optional).
|
|
||||||
- `GET /invoices/{id}/` invoice detail.
|
|
||||||
- `GET /invoices/{id}/download_pdf/` returns PDF bytes (placeholder).
|
|
||||||
- Payment endpoints (`PaymentViewSet`):
|
|
||||||
- `GET /payments/` list payments (status filter optional).
|
|
||||||
- `GET /payment-methods/available/` returns available methods (country/config driven).
|
|
||||||
- `POST /payments/manual/` submit manual payment for an invoice (requires invoice_id, payment_method, transaction_reference; rejects already paid).
|
|
||||||
- Account payment methods (`AccountPaymentMethodViewSet`):
|
|
||||||
- CRUD at `/payment-methods/`; `POST /payment-methods/{id}/set_default/` toggles default for the account.
|
|
||||||
- Credit packages (`CreditPackageViewSet`):
|
|
||||||
- `GET /credit-packages/` list active packages.
|
|
||||||
- `POST /credit-packages/{id}/purchase/` creates invoice and returns next_action (Stripe/PayPal placeholders; manual fallback returns invoice info).
|
|
||||||
- Credit transactions (`CreditTransactionViewSet`):
|
|
||||||
- `GET /transactions/` ledger of credit changes; also registered under `/credits/transactions/`.
|
|
||||||
- Credit balance/usage/limits (`modules/billing/views.py`):
|
|
||||||
- `GET /credits/balance/` returns credits, plan monthly credits, month-to-date credits used.
|
|
||||||
- `GET /credits/usage/` returns paginated usage logs with filters; `GET /credits/usage/summary/` aggregates by operation/model and totals; `GET /credits/usage/limits/` returns plan/account limits and usage counts.
|
|
||||||
- Permissions/tenancy: authenticated users; account-scoped querysets; viewsets rely on `request.account` or `request.user.account`.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `Invoice`, `Payment`, `CreditPackage`, `CreditTransaction`, `AccountPaymentMethod`, `PaymentMethodConfig`, `CreditUsageLog`, `CreditCostConfig`, `Account`, `Plan`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Router dispatch → viewset → service calls (invoice/payment/credit) → DB updates → unified responses. Balance/usage endpoints compute aggregates on demand.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Credit costs/checks used by writer/automation AI flows; balances/usage reflect those deductions.
|
|
||||||
- Plan limits surfaced in limits endpoint relate to account management in auth/account domains.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Invoices: draft/pending/paid/void/uncollectible; Payments: pending/pending_approval/processing/succeeded/completed/failed/refunded/cancelled; Credits balance changes via transactions.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Manual payment: 400 on missing fields or already-paid invoice.
|
|
||||||
- Credit operations: 402 returned by upstream callers on `InsufficientCreditsError`.
|
|
||||||
- Balance/usage: returns zeros/empty when account/plan missing.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All billing data filtered by account; no cross-tenant access. Default viewsets inherit account scoping from base classes.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- Credit costs come from `CreditCostConfig` or constants; deductions recorded in ledger + usage logs.
|
|
||||||
- Available methods and packages are filtered by active flags/config.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- None specific; Stripe/PayPal integrations are placeholders; any future webhooks should update invoices/payments/credits accordingly.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Ledger + usage logs kept in sync via `CreditService`.
|
|
||||||
- Manual payment flow avoids storing sensitive data; account payment methods store metadata only.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Use `CreditService` for credit math; do not mutate balances directly.
|
|
||||||
- Implement Stripe/PayPal flows in `PaymentService` if adding gateways; wire webhooks to update models.
|
|
||||||
- Keep account scoping on all queries; reuse existing viewsets for new billing features.
|
|
||||||
|
|
||||||
266
docs/20-API/ENDPOINTS.md
Normal file
266
docs/20-API/ENDPOINTS.md
Normal file
@@ -0,0 +1,266 @@
|
|||||||
|
# API Endpoints Reference
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Base URL:** `/api/v1/`
|
||||||
|
**Documentation:** `/api/docs/` (Swagger) | `/api/redoc/` (ReDoc)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Authentication
|
||||||
|
|
||||||
|
All endpoints require authentication unless noted.
|
||||||
|
|
||||||
|
**Methods:**
|
||||||
|
- `Authorization: Bearer <jwt_token>` - JWT token
|
||||||
|
- `Authorization: ApiKey <key>` - API key (WordPress integration)
|
||||||
|
- Session cookie (Django Admin)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Auth Endpoints (`/api/v1/auth/`)
|
||||||
|
|
||||||
|
| Method | Path | Handler | Auth | Purpose |
|
||||||
|
|--------|------|---------|------|---------|
|
||||||
|
| POST | `/register/` | `RegisterView` | ❌ | Create account |
|
||||||
|
| POST | `/login/` | `LoginView` | ❌ | Get tokens |
|
||||||
|
| POST | `/logout/` | `LogoutView` | ✅ | Invalidate session |
|
||||||
|
| POST | `/token/refresh/` | `RefreshTokenView` | ✅ | Refresh access token |
|
||||||
|
| POST | `/password/change/` | `ChangePasswordView` | ✅ | Change password |
|
||||||
|
| POST | `/password/reset/` | `RequestPasswordResetView` | ❌ | Request reset email |
|
||||||
|
| POST | `/password/reset/confirm/` | `ResetPasswordView` | ❌ | Confirm reset |
|
||||||
|
| GET | `/groups/` | `RoleViewSet.list` | ✅ | List roles |
|
||||||
|
| GET | `/users/` | `UserViewSet.list` | ✅ | List users |
|
||||||
|
| POST | `/users/` | `UserViewSet.create` | ✅ | Create user |
|
||||||
|
| GET | `/account/` | `AccountViewSet.retrieve` | ✅ | Get account |
|
||||||
|
| PUT | `/account/` | `AccountViewSet.update` | ✅ | Update account |
|
||||||
|
| GET | `/sites/` | `SiteViewSet.list` | ✅ | List sites |
|
||||||
|
| POST | `/sites/` | `SiteViewSet.create` | ✅ | Create site |
|
||||||
|
| GET | `/sectors/` | `SectorViewSet.list` | ✅ | List sectors |
|
||||||
|
| POST | `/sectors/` | `SectorViewSet.create` | ✅ | Create sector |
|
||||||
|
| GET | `/industries/` | `IndustryViewSet.list` | ✅ | List industries |
|
||||||
|
| GET | `/seed-keywords/` | `SeedKeywordViewSet.list` | ✅ | List seed keywords |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planner Endpoints (`/api/v1/planner/`)
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/keywords/` | `KeywordViewSet.list` | List keywords |
|
||||||
|
| POST | `/keywords/` | `KeywordViewSet.create` | Create keyword |
|
||||||
|
| GET | `/keywords/{id}/` | `KeywordViewSet.retrieve` | Get keyword |
|
||||||
|
| PUT | `/keywords/{id}/` | `KeywordViewSet.update` | Update keyword |
|
||||||
|
| DELETE | `/keywords/{id}/` | `KeywordViewSet.destroy` | Delete keyword |
|
||||||
|
| POST | `/keywords/bulk_delete/` | `KeywordViewSet.bulk_delete` | Hard delete multiple |
|
||||||
|
| POST | `/keywords/bulk_status/` | `KeywordViewSet.bulk_status` | Update status |
|
||||||
|
| POST | `/keywords/add_to_workflow/` | `KeywordViewSet.add_to_workflow` | Add seed keywords |
|
||||||
|
| GET | `/clusters/` | `ClusterViewSet.list` | List clusters |
|
||||||
|
| POST | `/clusters/` | `ClusterViewSet.create` | Create cluster |
|
||||||
|
| POST | `/clusters/auto_cluster/` | `ClusterViewSet.auto_cluster` | AI clustering |
|
||||||
|
| POST | `/clusters/generate_ideas/` | `ClusterViewSet.generate_ideas` | Generate ideas |
|
||||||
|
| GET | `/ideas/` | `ContentIdeaViewSet.list` | List ideas |
|
||||||
|
| POST | `/ideas/` | `ContentIdeaViewSet.create` | Create idea |
|
||||||
|
| POST | `/ideas/create_tasks/` | `ContentIdeaViewSet.create_tasks` | Convert to tasks |
|
||||||
|
|
||||||
|
**Query Parameters:**
|
||||||
|
- `?site_id=` - Filter by site
|
||||||
|
- `?sector_id=` - Filter by sector
|
||||||
|
- `?status=` - Filter by status
|
||||||
|
- `?cluster_id=` - Filter by cluster
|
||||||
|
- `?difficulty_min=`, `?difficulty_max=` - Difficulty range
|
||||||
|
- `?volume_min=`, `?volume_max=` - Volume range
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Writer Endpoints (`/api/v1/writer/`)
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/tasks/` | `TaskViewSet.list` | List tasks |
|
||||||
|
| POST | `/tasks/` | `TaskViewSet.create` | Create task |
|
||||||
|
| POST | `/tasks/bulk_create/` | `TaskViewSet.bulk_create` | Create multiple |
|
||||||
|
| POST | `/tasks/{id}/generate_content/` | `TaskViewSet.generate_content` | AI generation |
|
||||||
|
| GET | `/content/` | `ContentViewSet.list` | List content |
|
||||||
|
| POST | `/content/` | `ContentViewSet.create` | Create content |
|
||||||
|
| GET | `/content/{id}/` | `ContentViewSet.retrieve` | Get content |
|
||||||
|
| PUT | `/content/{id}/` | `ContentViewSet.update` | Update content |
|
||||||
|
| POST | `/content/{id}/update_content/` | `ContentViewSet.update_content` | Update with validation |
|
||||||
|
| POST | `/content/{id}/generate_images/` | `ContentViewSet.generate_images` | Generate images |
|
||||||
|
| POST | `/content/{id}/publish_to_wordpress/` | `ContentViewSet.publish_to_wordpress` | Publish to WP |
|
||||||
|
| GET | `/images/` | `ImageViewSet.list` | List images |
|
||||||
|
| POST | `/images/generate_for_content/` | `ImageViewSet.generate_for_content` | Generate images |
|
||||||
|
| POST | `/images/regenerate/` | `ImageViewSet.regenerate` | Regenerate image |
|
||||||
|
| GET | `/taxonomies/` | `TaxonomyViewSet.list` | List taxonomies |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Billing Endpoints (`/api/v1/billing/`)
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/balance/` | `CreditBalanceViewSet.list` | Current balance |
|
||||||
|
| GET | `/usage/` | `CreditUsageViewSet.list` | Usage log |
|
||||||
|
| GET | `/usage/summary/` | `CreditUsageViewSet.summary` | Usage summary |
|
||||||
|
| GET | `/usage/limits/` | `CreditUsageViewSet.limits` | Plan limits |
|
||||||
|
| GET | `/transactions/` | `TransactionViewSet.list` | Transaction history |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Integration Endpoints (`/api/v1/integration/`)
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/` | `SiteIntegrationViewSet.list` | List integrations |
|
||||||
|
| POST | `/` | `SiteIntegrationViewSet.create` | Create integration |
|
||||||
|
| PUT | `/{id}/` | `SiteIntegrationViewSet.update` | Update integration |
|
||||||
|
| DELETE | `/{id}/` | `SiteIntegrationViewSet.destroy` | Delete integration |
|
||||||
|
| POST | `/{id}/test_connection/` | Test connection | Verify credentials |
|
||||||
|
| POST | `/{id}/test_collection_connection/` | Test collection | Test specific type |
|
||||||
|
| POST | `/{id}/sync/` | Trigger sync | Start sync |
|
||||||
|
| GET | `/{id}/sync_status/` | Sync status | Current progress |
|
||||||
|
| POST | `/{id}/update_structure/` | Update structure | Refresh site data |
|
||||||
|
| GET | `/{id}/content_types/` | Content types | Available types |
|
||||||
|
| GET | `/{id}/sync_health/` | Sync health | Statistics |
|
||||||
|
| POST | `/site_sync/` | Site-level sync | Sync by site ID |
|
||||||
|
| POST | `/webhook/wordpress/` | WordPress webhook | Receive WP updates |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## System Endpoints (`/api/v1/system/`)
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/settings/integrations/openai/` | Get OpenAI settings | Current config |
|
||||||
|
| PUT | `/settings/integrations/openai/` | Save OpenAI settings | Update config |
|
||||||
|
| GET | `/settings/integrations/image_generation/` | Get image settings | Current config |
|
||||||
|
| PUT | `/settings/integrations/image_generation/` | Save image settings | Update config |
|
||||||
|
| POST | `/settings/integrations/test/` | Test connection | Verify API keys |
|
||||||
|
| GET | `/prompts/` | List prompts | All prompts |
|
||||||
|
| GET | `/prompts/{type}/` | Get prompt | Specific prompt |
|
||||||
|
| PUT | `/prompts/{type}/` | Save prompt | Update prompt |
|
||||||
|
| POST | `/prompts/{type}/reset/` | Reset prompt | Reset to default |
|
||||||
|
| GET | `/modules/` | Get modules | Module enable state |
|
||||||
|
| PUT | `/modules/` | Save modules | Update enabled |
|
||||||
|
| GET | `/health/` | Health check | System status |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Automation Endpoints (`/api/v1/automation/`)
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/config/` | Get config | Automation config |
|
||||||
|
| PUT | `/update_config/` | Update config | Save settings |
|
||||||
|
| POST | `/run_now/` | Run now | Start manual run |
|
||||||
|
| GET | `/current_run/` | Current run | Run status |
|
||||||
|
| GET | `/pipeline_overview/` | Pipeline | Stage counts |
|
||||||
|
| GET | `/current_processing/` | Processing | Live status |
|
||||||
|
| POST | `/pause/` | Pause | Pause run |
|
||||||
|
| POST | `/resume/` | Resume | Resume run |
|
||||||
|
| POST | `/cancel/` | Cancel | Cancel run |
|
||||||
|
| GET | `/history/` | History | Past runs |
|
||||||
|
| GET | `/logs/` | Logs | Activity log |
|
||||||
|
| GET | `/estimate/` | Estimate | Credit estimate |
|
||||||
|
|
||||||
|
**Query Parameters:** All require `?site_id=`, run-specific require `?run_id=`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Linker Endpoints (`/api/v1/linker/`)
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| POST | `/process/` | `LinkerViewSet.process` | Process single content |
|
||||||
|
| POST | `/batch_process/` | `LinkerViewSet.batch_process` | Process multiple |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Optimizer Endpoints (`/api/v1/optimizer/`)
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| POST | `/optimize/` | `OptimizerViewSet.optimize` | Optimize content |
|
||||||
|
| POST | `/batch_optimize/` | `OptimizerViewSet.batch_optimize` | Batch optimize |
|
||||||
|
| POST | `/analyze/` | `OptimizerViewSet.analyze` | Analyze only |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Publisher Endpoints (`/api/v1/publisher/`)
|
||||||
|
|
||||||
|
| Method | Path | Handler | Purpose |
|
||||||
|
|--------|------|---------|---------|
|
||||||
|
| GET | `/records/` | `PublishingRecordViewSet.list` | List records |
|
||||||
|
| POST | `/records/` | `PublishingRecordViewSet.create` | Create record |
|
||||||
|
| GET | `/deployments/` | `DeploymentViewSet.list` | List deployments |
|
||||||
|
| POST | `/publish/` | `PublishContentViewSet.publish` | Publish content |
|
||||||
|
| GET | `/publish/status/` | `PublishContentViewSet.status` | Publish status |
|
||||||
|
| GET | `/site-definition/` | `SiteDefinitionViewSet.list` | Public site def |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Response Format
|
||||||
|
|
||||||
|
### Success Response
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"success": true,
|
||||||
|
"data": { ... },
|
||||||
|
"message": "Optional message"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Error Response
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"success": false,
|
||||||
|
"error": "Error message",
|
||||||
|
"code": "ERROR_CODE"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Paginated Response
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"count": 100,
|
||||||
|
"next": "http://api.igny8.com/api/v1/planner/keywords/?page=2",
|
||||||
|
"previous": null,
|
||||||
|
"results": [ ... ]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## HTTP Status Codes
|
||||||
|
|
||||||
|
| Code | Meaning |
|
||||||
|
|------|---------|
|
||||||
|
| 200 | Success |
|
||||||
|
| 201 | Created |
|
||||||
|
| 400 | Bad Request |
|
||||||
|
| 401 | Unauthorized |
|
||||||
|
| 402 | Payment Required (insufficient credits) |
|
||||||
|
| 403 | Forbidden |
|
||||||
|
| 404 | Not Found |
|
||||||
|
| 429 | Rate Limited |
|
||||||
|
| 500 | Server Error |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Rate Limiting
|
||||||
|
|
||||||
|
Rate limits are scoped by operation type:
|
||||||
|
|
||||||
|
| Scope | Limit |
|
||||||
|
|-------|-------|
|
||||||
|
| AI operations | 60/min |
|
||||||
|
| Content operations | 100/min |
|
||||||
|
| Auth operations | 20/min |
|
||||||
|
| General | 300/min |
|
||||||
|
|
||||||
|
Rate limit headers included in responses:
|
||||||
|
- `X-RateLimit-Limit`
|
||||||
|
- `X-RateLimit-Remaining`
|
||||||
|
- `X-RateLimit-Reset`
|
||||||
@@ -1,66 +0,0 @@
|
|||||||
# Integration Endpoints
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Document integration API endpoints for managing site integrations, testing connections, syncing, and handling WordPress webhooks.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Routing: `backend/igny8_core/modules/integration/urls.py`
|
|
||||||
- Views: `backend/igny8_core/modules/integration/views.py`
|
|
||||||
- Webhooks: `backend/igny8_core/modules/integration/webhooks.py`
|
|
||||||
- Services: `backend/igny8_core/business/integration/services/*`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- CRUD for `SiteIntegration` records.
|
|
||||||
- Test connections (detail and collection-level), trigger syncs, fetch sync status, update site structure, and sync WordPress content.
|
|
||||||
- Expose WordPress webhook endpoints for status/metadata updates.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Base path: `/api/v1/integration/`
|
|
||||||
- Viewset: `IntegrationViewSet` (`/integrations/`, inherits `SiteSectorModelViewSet` but overrides queryset to filter by site_id since integrations have site only).
|
|
||||||
- Permissions: `IsAuthenticatedAndActive`, `IsEditorOrAbove`; throttle scope `integration`.
|
|
||||||
- Serializer exposes derived `api_key` (from credentials) and validates WordPress integrations require API key.
|
|
||||||
- Actions:
|
|
||||||
- `test_connection` (detail): calls `IntegrationService.test_connection` on existing integration; returns success/error.
|
|
||||||
- `test_connection_collection` (POST `integrations/test-connection/`, AllowAny): accepts site_id/api_key/site_url; validates site ownership or matching `Site.wp_api_key`; creates integration if missing; tests WordPress connection; deletes integration on failure; returns integration_id on success.
|
|
||||||
- `sync`: triggers `SyncService.sync` (direction metadata/both/to_external/from_external); metadata-only path uses `SyncMetadataService` internally.
|
|
||||||
- `sync_status`: returns sync status from `SyncService`.
|
|
||||||
- `update_site_structure`: updates `config_json.content_types` (post_types/taxonomies, plugin flags, timestamp) from WordPress push payload.
|
|
||||||
- `get_content_types`: returns content type config from integration config.
|
|
||||||
- `debug_status`: returns debug data via `SyncHealthService`.
|
|
||||||
- `sync_wordpress_content`: calls `ContentSyncService.sync_from_wordpress` for a specific content_id.
|
|
||||||
- Webhooks:
|
|
||||||
- `/webhooks/wordpress/status/` and `/webhooks/wordpress/metadata/` endpoints for WordPress callbacks (handled in `webhooks.py`).
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `SiteIntegration`, `SyncEvent`, `Site`, `Account`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- CRUD/test/sync requests → DRF auth → site-filtered queryset → service calls → model updates/sync actions. Collection-level test may create/delete integration based on result. Webhooks call handlers for incoming WordPress events.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Integrations used by publishing/sync flows to external platforms; WordPress API key path relies on `Site.wp_api_key` and integration credentials.
|
|
||||||
- Sync services interact with writer/planner content for metadata/content sync.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Integration `sync_status/last_sync_at/is_active/sync_enabled` updated by sync/test actions; `SyncEvent` records per-action success/failure with timestamps and IDs.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Invalid site/auth/API key return 403/404/400; unsupported platforms raise `NotImplementedError`; failed collection-level tests delete newly created integrations.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All endpoints scoped to the site/account; queryset filters by site_id and permissions enforce authenticated editor-or-above unless using collection-level API key path.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- None; integration endpoints do not change credits.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- None specific; sync logic executes in service calls (may delegate internally).
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Single integration per site+platform; collection-level test supports plugin onboarding with API key.
|
|
||||||
- Credentials stored as JSON (encryption pending).
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Use `IntegrationService` for CRUD/test; extend platform support via new test logic and serializer validation.
|
|
||||||
- Keep site scoping and API key validation intact for collection-level tests.
|
|
||||||
|
|
||||||
@@ -1,62 +0,0 @@
|
|||||||
# Planner Endpoints
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Document planner API endpoints for keywords, clusters, and content ideas, including filters, bulk actions, and validation rules.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Routing: `backend/igny8_core/modules/planner/urls.py`
|
|
||||||
- Views: `backend/igny8_core/modules/planner/views.py`
|
|
||||||
- Serializers: `backend/igny8_core/modules/planner/serializers.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- CRUD for planner entities scoped by account/site/sector with search, filtering, ordering, and bulk operations.
|
|
||||||
- Enforce site/sector requirements and seed keyword alignment on creation.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Base path: `/api/v1/planner/`
|
|
||||||
- Viewsets (all inherit `SiteSectorModelViewSet`, scoped by account/site/sector, throttled with scope `planner`, paginated):
|
|
||||||
- `KeywordViewSet` (`/keywords/`):
|
|
||||||
- Search: `seed_keyword__keyword`.
|
|
||||||
- Filters: status, cluster_id, seed_keyword intent/id; custom difficulty_min/max and volume_min/max (uses overrides or seed values).
|
|
||||||
- Ordering: created_at, seed volume/difficulty.
|
|
||||||
- Bulk: `bulk_delete`, `bulk_update` (status), `bulk_add_from_seed` (create Keywords from SeedKeywords after validating site/sector).
|
|
||||||
- Create requires site_id and sector_id; validates site/sector existence and alignment.
|
|
||||||
- `ClusterViewSet` (`/clusters/`):
|
|
||||||
- Filters: status; standard CRUD; site/sector required on create.
|
|
||||||
- `ContentIdeasViewSet` (`/ideas/`):
|
|
||||||
- Filters: status, cluster; standard CRUD; site/sector required on create.
|
|
||||||
- Serializers enforce required fields on create (`KeywordSerializer` requires seed_keyword_id; site/sector write-only), expose read-only seed-derived fields and cluster/sector names.
|
|
||||||
- Error handling: unified responses; validation errors for missing site/sector/seed; bulk actions return 400 when IDs/status missing.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `Keywords`, `Clusters`, `ContentIdeas`, `SeedKeyword`, plus `Account`, `Site`, `Sector`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Requests → DRF auth → base viewset filters (account/site/sector) → serializer validation → model save → unified response; bulk actions operate on filtered queryset.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Feeds automation Stage 1–3 and writer tasks/content generation.
|
|
||||||
- Billing credits may be checked upstream when AI clustering/idea generation is invoked.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Keywords: `new` → `mapped`; Clusters: `new` → `mapped`; ContentIdeas: `new` → `queued` → `completed`.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Missing required IDs or validation failures return 400; unified 500 on unhandled errors.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All endpoints scoped to account/site/sector via base viewset; admin/developer/system roles can bypass filtering, but data retains tenant/site/sector fields.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- No direct deductions in these endpoints; AI clustering/idea generation costs enforced where services are called.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- None in planner endpoints; automation handles async flows.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Strict site/sector requirements prevent cross-site contamination.
|
|
||||||
- Seed linkage keeps keyword metadata consistent; overrides allow site-specific tuning.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Supply site_id/sector_id on create; use bulk actions for batch changes; extend filters/search/order as needed while respecting base scoping.
|
|
||||||
|
|
||||||
@@ -1,68 +0,0 @@
|
|||||||
# Writer Endpoints
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Document writer API endpoints for tasks, content, images, and taxonomies, including bulk actions and AI content generation triggers.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Routing: `backend/igny8_core/modules/writer/urls.py`
|
|
||||||
- Views: `backend/igny8_core/modules/writer/views.py`
|
|
||||||
- Serializers: `backend/igny8_core/modules/writer/serializers.py`
|
|
||||||
- Services: `backend/igny8_core/business/content/services/content_generation_service.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- CRUD for writer entities scoped by account/site/sector.
|
|
||||||
- Provide bulk operations and AI content generation for tasks.
|
|
||||||
- Manage images and taxonomy terms linked to content/tasks.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Base path: `/api/v1/writer/`
|
|
||||||
- Viewsets (inherit `SiteSectorModelViewSet`, throttled `writer`, paginated):
|
|
||||||
- `TasksViewSet` (`/tasks/`):
|
|
||||||
- Filters: status, cluster_id, content_type, content_structure; search title/keywords; ordering by title/created_at/status.
|
|
||||||
- Bulk: `bulk_delete`, `bulk_update` (status).
|
|
||||||
- `auto_generate_content`: accepts task IDs (max 10), requires account, enforces credit checks via `ContentGenerationService`, returns async task_id or synchronous result; 402 on insufficient credits.
|
|
||||||
- Create requires site_id/sector_id; validates site/sector alignment; sets account/site/sector explicitly.
|
|
||||||
- `ImagesViewSet` (`/images/`):
|
|
||||||
- Filters: task_id, content_id, image_type, status; ordering by created_at/position/id.
|
|
||||||
- perform_create enforces site/sector presence (from request or defaults), sets account/site/sector; serves local files via `file` action with existence checks.
|
|
||||||
- `ContentViewSet` (`/content/`):
|
|
||||||
- Filters/order/search defined in viewset (titles, status, etc.); manages taxonomy relationships; exposes status, SEO fields, cluster link, external IDs/URLs.
|
|
||||||
- `ContentTaxonomyViewSet` (`/taxonomies/`):
|
|
||||||
- CRUD for taxonomy terms (category/tag) scoped to site/sector.
|
|
||||||
- Serializers enforce required fields on create (e.g., cluster/content_type/content_structure for tasks; site/sector for content) and expose read-only derived fields (cluster/sector names, image/taxonomy info).
|
|
||||||
- Error handling: unified responses; validation errors return 400; missing resources return 404; `auto_generate_content` can return 402/500 on credits/other errors.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `Tasks`, `Content`, `Images`, `ContentTaxonomy`, and related domain models; `Account`, `Site`, `Sector`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Requests → DRF auth → base viewset filtering (account/site/sector) → serializer validation → model save → unified response; bulk/AI actions invoke services.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Planner clusters/ideas feed tasks; automation stages 3–6 operate on writer data; publishing/integration uses content/taxonomies/images.
|
|
||||||
- Billing credits enforced during AI content generation via `ContentGenerationService`.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Tasks: `queued` → `completed`; Content: `draft` → `review` → `published`; Images: `pending` → `generated`.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Standard unified responses; credit errors mapped to 402 in `auto_generate_content`; file serving returns 404 for missing images.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All endpoints scoped to account/site/sector; base viewset and permissions enforce access; admin/developer/system may bypass filtering but records retain tenant/site/sector fields.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- AI generation actions check/deduct credits via billing services; other CRUD does not alter credits.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- `auto_generate_content` may enqueue Celery AI tasks when returning task_id; automation uses writer endpoints indirectly via services.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Strict site/sector validation on create; max 10 tasks per auto_generate_content to manage load/credits.
|
|
||||||
- Image file serving is permissive but guarded by existence checks.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Supply site_id/sector_id on create; use bulk actions for batch changes.
|
|
||||||
- When extending AI flows, keep credit checks and account/site scoping intact.
|
|
||||||
- Reuse existing viewsets/serializers for new fields; ensure filters/throttles stay aligned with module scope.
|
|
||||||
|
|
||||||
@@ -1,76 +0,0 @@
|
|||||||
# Global UI Components and Providers
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe global layout, guards, and utility components/providers used throughout the frontend.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- App composition and routing: `frontend/src/App.tsx`
|
|
||||||
- Entry/providers: `frontend/src/main.tsx`
|
|
||||||
- Layout: `frontend/src/layout/AppLayout.tsx`, `frontend/src/layout/AppSidebar.tsx`, `frontend/src/layout/AppHeader.tsx` (and related layout files)
|
|
||||||
- Guards: `frontend/src/components/auth/ProtectedRoute.tsx`, `frontend/src/components/common/ModuleGuard.tsx`
|
|
||||||
- Global utilities: `frontend/src/components/common/ScrollToTop.tsx`, `frontend/src/components/common/GlobalErrorDisplay.tsx`, `frontend/src/components/common/LoadingStateMonitor.tsx`, `frontend/src/components/common/ErrorBoundary.tsx`
|
|
||||||
- Providers: `frontend/src/context/ThemeContext.tsx`, `frontend/src/context/HeaderMetricsContext.tsx`, `frontend/src/components/ui/toast/ToastContainer.tsx`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Wrap the app with error handling, theming, metrics, toasts, and routing.
|
|
||||||
- Enforce authentication and module access at the route level.
|
|
||||||
- Provide global UI behaviors (scroll reset, error banner, loading monitor).
|
|
||||||
- Supply consistent layout (sidebar/header/content) for protected areas.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Providers (`main.tsx`):
|
|
||||||
- `ErrorBoundary` wraps the entire app.
|
|
||||||
- `ThemeProvider` supplies theme context.
|
|
||||||
- `HeaderMetricsProvider` supplies header metrics context.
|
|
||||||
- `ToastProvider` exposes toast notifications.
|
|
||||||
- `BrowserRouter` provides routing; renders `<App />`.
|
|
||||||
- Routing shell (`App.tsx`):
|
|
||||||
- `GlobalErrorDisplay` renders global errors; `LoadingStateMonitor` tracks loading states.
|
|
||||||
- `ScrollToTop` resets scroll on route changes.
|
|
||||||
- Public routes: `/signin`, `/signup`.
|
|
||||||
- Protected routes: wrapped in `ProtectedRoute` → `AppLayout`; `ModuleGuard` used per-module.
|
|
||||||
- Lazy-loaded module pages inside routes; ModuleGuard enforces module access flags.
|
|
||||||
- Guards:
|
|
||||||
- `ProtectedRoute`: checks `useAuthStore` for authentication; redirects unauthenticated users to sign-in; logs out on failed refresh in App effect.
|
|
||||||
- `ModuleGuard`: gates module pages based on module enable settings/permissions.
|
|
||||||
- Layout:
|
|
||||||
- `AppLayout` composes sidebar/header/content; `AppSidebar` uses auth store for user/nav; header components provide top-level actions and metrics.
|
|
||||||
- Utilities:
|
|
||||||
- `ScrollToTop` listens to route changes and scrolls to top.
|
|
||||||
- `GlobalErrorDisplay` shows global error banners.
|
|
||||||
- `LoadingStateMonitor` tracks loading indicators globally.
|
|
||||||
- `ToastProvider` supplies toast UI primitives for notifications.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Context values from theme, header metrics, toast providers; auth state from `authStore`; module enable settings from `settingsStore`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- App bootstrap wraps providers → routes render → ProtectedRoute checks auth → ModuleGuard checks module access → layout renders with sidebar/header → pages load lazily and fetch data.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Auth/module settings drive guards; toasts/errors/loading monitors are available to all pages; layout navigation links modules.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- `ErrorBoundary` catches render errors.
|
|
||||||
- `GlobalErrorDisplay` surfaces application-level errors.
|
|
||||||
- `ProtectedRoute` logs out on failed refresh in App effect; unauthorized users are redirected.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Enforced via backend auth and module enable settings; guards rely on auth/module settings to gate access.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- None in UI components; billing info shown in pages that consume billing store/endpoints.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- None; components react to store state and router changes.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Provider stack covers theme, metrics, toasts, error boundary, routing.
|
|
||||||
- Guards ensure unauthorized access is blocked at the route level.
|
|
||||||
- Global utilities avoid repeated boilerplate for scroll/error/loading behaviors.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Add new protected pages under `ProtectedRoute` and wrap with `ModuleGuard` when module-scoped.
|
|
||||||
- Use existing toast/error/loading utilities instead of duplicating.
|
|
||||||
- Keep provider order consistent (ErrorBoundary → Theme/Header/Toast → Router → App).
|
|
||||||
|
|
||||||
@@ -1,84 +0,0 @@
|
|||||||
# Frontend Architecture
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Explain the frontend structure, routing, state management, providers, and module organization in the React/Vite app.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Entry: `frontend/src/main.tsx`
|
|
||||||
- Root app/routing: `frontend/src/App.tsx`
|
|
||||||
- State (Zustand stores): `frontend/src/store/*` (authStore, siteStore, billingStore, settingsStore, onboardingStore, columnVisibilityStore, pageSizeStore)
|
|
||||||
- API layer: `frontend/src/api/*` and `frontend/src/services/api.ts`
|
|
||||||
- Layout and components: `frontend/src/layout/*`, `frontend/src/components/*`
|
|
||||||
- Pages: `frontend/src/pages/*` (modules and settings)
|
|
||||||
- Config: `frontend/src/config/*`
|
|
||||||
- Styles: `frontend/src/index.css`, `frontend/src/styles/igny8-colors.css`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Provide SPA routing with protected routes and module guards.
|
|
||||||
- Maintain global state for auth, site context, billing, settings, UI preferences via Zustand.
|
|
||||||
- Lazy-load module pages to optimize initial load.
|
|
||||||
- Wrap the app with providers for theming, toasts, header metrics, error boundary, and router.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Entry (`main.tsx`):
|
|
||||||
- Imports global styles and vendor CSS (Swiper, Flatpickr).
|
|
||||||
- Wraps the app with `ErrorBoundary`, `ThemeProvider`, `HeaderMetricsProvider`, `ToastProvider`, `BrowserRouter`, then renders `<App />` via React 18 `createRoot`.
|
|
||||||
- Routing (`App.tsx`):
|
|
||||||
- Uses `BrowserRouter` (in main) with `<Routes>`/`<Route>` inside App.
|
|
||||||
- Public routes: `/signin`, `/signup`.
|
|
||||||
- Protected routes wrapped by `ProtectedRoute` → `AppLayout`; `ModuleGuard` used per-module.
|
|
||||||
- Default dashboard `/` loads `Dashboard/Home`.
|
|
||||||
- Planner routes redirect `/planner` → `/planner/keywords`; writer routes redirect `/writer` → `/writer/tasks`.
|
|
||||||
- Extensive lazy-loaded pages for Planner, Writer, Automation, Linker, Optimizer, Thinker (system), Billing, Admin, Settings, Sites, Help, Reference, Components, UI elements, etc.
|
|
||||||
- `GlobalErrorDisplay` and `LoadingStateMonitor` mounted globally; `ScrollToTop` for navigation changes.
|
|
||||||
- Auth refresh: on mount, if authenticated with token, calls `refreshUser`; on failure with missing credentials message, ignores; otherwise logs out.
|
|
||||||
- State (Zustand examples):
|
|
||||||
- `authStore`: persists user/token/refreshToken; `login`/`register` hit `/api/v1/auth/...`; enforces account and active plan presence; `refreshUser` fetches user from `/api/v1/auth/users/me/`; `refreshToken` hits `/api/v1/auth/refresh/`; logout clears all auth fields.
|
|
||||||
- `siteStore`: stores current site/sector and list; used for context across modules.
|
|
||||||
- `billingStore`, `settingsStore`, `onboardingStore`, `columnVisibilityStore`, `pageSizeStore`: manage billing data, settings flags, onboarding steps, table visibility, pagination sizes.
|
|
||||||
- API layer:
|
|
||||||
- `services/api.ts` provides `fetchAPI` helper for authenticated calls (uses token from auth store, handles JSON, throws on non-OK).
|
|
||||||
- `api/*` modules define typed client calls per domain (auth, planner, writer, etc.).
|
|
||||||
- Layout/components:
|
|
||||||
- `AppLayout` wraps sidebar/header/content; `ModuleGuard` enforces module access flags; `ProtectedRoute` enforces auth.
|
|
||||||
- UI utilities: toasts, error boundary, global loading monitor, scroll-to-top, etc.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Frontend state shapes defined in Zustand stores (auth user, site context, billing info, settings).
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- App bootstrap → providers → App routes → ProtectedRoute checks auth → ModuleGuard checks module access → lazy-loaded pages fetch data via `api/*` using tokens from authStore → state updates via stores.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Auth store tokens used by fetchAPI for all modules.
|
|
||||||
- Site store context influences planner/writer/automation calls.
|
|
||||||
- Billing store interacts with billing endpoints; settings store influences UI/features; onboarding/table stores affect page rendering.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Auth: login/register → set user/token; refresh → update tokens/user; logout clears.
|
|
||||||
- Site context changes update current site/sector for downstream calls.
|
|
||||||
- Billing/settings/onboarding/table state mutate via respective store actions.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- `GlobalErrorDisplay` shows errors; `LoadingStateMonitor` tracks loading; `ProtectedRoute` logs out on failed refresh.
|
|
||||||
- `fetchAPI` throws on non-OK; authStore catches and surfaces messages; ModuleGuard/ProtectedRoute handle unauthorized access.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Enforced via tokens and site context; ModuleGuard and ProtectedRoute rely on backend responses (account/plan) and module flags.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- Auth store enforces active plan on login; billing store/pages call billing endpoints for balances/transactions/purchases.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- None on frontend; relies on backend/Celery for async work; uses Suspense for lazy loading.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Lazy-loaded routes minimize initial bundle.
|
|
||||||
- Zustand persistence keeps auth/session across reloads.
|
|
||||||
- Providers wrap app for theming, toasts, error boundary to improve UX stability.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Add routes inside `App.tsx` under ProtectedRoute for authenticated pages; use ModuleGuard for module-gated pages.
|
|
||||||
- Create new stores in `src/store` for shared state; use `fetchAPI` for authenticated requests.
|
|
||||||
- Keep lazy loading for heavy pages and maintain Suspense fallbacks where needed.
|
|
||||||
|
|
||||||
@@ -1,76 +0,0 @@
|
|||||||
# Global UI Components and Providers
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe global layout, guards, and utility components/providers used throughout the frontend.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- App composition and routing: `frontend/src/App.tsx`
|
|
||||||
- Entry/providers: `frontend/src/main.tsx`
|
|
||||||
- Layout: `frontend/src/layout/AppLayout.tsx`, `frontend/src/layout/AppSidebar.tsx`, `frontend/src/layout/AppHeader.tsx` (and related layout files)
|
|
||||||
- Guards: `frontend/src/components/auth/ProtectedRoute.tsx`, `frontend/src/components/common/ModuleGuard.tsx`
|
|
||||||
- Global utilities: `frontend/src/components/common/ScrollToTop.tsx`, `frontend/src/components/common/GlobalErrorDisplay.tsx`, `frontend/src/components/common/LoadingStateMonitor.tsx`, `frontend/src/components/common/ErrorBoundary.tsx`
|
|
||||||
- Providers: `frontend/src/context/ThemeContext.tsx`, `frontend/src/context/HeaderMetricsContext.tsx`, `frontend/src/components/ui/toast/ToastContainer.tsx`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Wrap the app with error handling, theming, metrics, toasts, and routing.
|
|
||||||
- Enforce authentication and module access at the route level.
|
|
||||||
- Provide global UI behaviors (scroll reset, error banner, loading monitor).
|
|
||||||
- Supply consistent layout (sidebar/header/content) for protected areas.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Providers (`main.tsx`):
|
|
||||||
- `ErrorBoundary` wraps the entire app.
|
|
||||||
- `ThemeProvider` supplies theme context.
|
|
||||||
- `HeaderMetricsProvider` supplies header metrics context.
|
|
||||||
- `ToastProvider` exposes toast notifications.
|
|
||||||
- `BrowserRouter` provides routing; renders `<App />`.
|
|
||||||
- Routing shell (`App.tsx`):
|
|
||||||
- `GlobalErrorDisplay` renders global errors; `LoadingStateMonitor` tracks loading states.
|
|
||||||
- `ScrollToTop` resets scroll on route changes.
|
|
||||||
- Public routes: `/signin`, `/signup`.
|
|
||||||
- Protected routes: wrapped in `ProtectedRoute` → `AppLayout`; `ModuleGuard` used per-module.
|
|
||||||
- Lazy-loaded module pages inside routes; ModuleGuard enforces module access flags.
|
|
||||||
- Guards:
|
|
||||||
- `ProtectedRoute`: checks `useAuthStore` for authentication; redirects unauthenticated users to sign-in; logs out on failed refresh in App effect.
|
|
||||||
- `ModuleGuard`: gates module pages based on module enable settings/permissions.
|
|
||||||
- Layout:
|
|
||||||
- `AppLayout` composes sidebar/header/content; `AppSidebar` uses auth store for user/nav; header components provide top-level actions and metrics.
|
|
||||||
- Utilities:
|
|
||||||
- `ScrollToTop` listens to route changes and scrolls to top.
|
|
||||||
- `GlobalErrorDisplay` shows global error banners.
|
|
||||||
- `LoadingStateMonitor` tracks loading indicators globally.
|
|
||||||
- `ToastProvider` supplies toast UI primitives for notifications.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Context values from theme, header metrics, toast providers; auth state from `authStore`; module enable settings from `settingsStore`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- App bootstrap wraps providers → routes render → ProtectedRoute checks auth → ModuleGuard checks module access → layout renders with sidebar/header → pages load lazily and fetch data.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Auth/module settings drive guards; toasts/errors/loading monitors are available to all pages; layout navigation links modules.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- `ErrorBoundary` catches render errors.
|
|
||||||
- `GlobalErrorDisplay` surfaces application-level errors.
|
|
||||||
- `ProtectedRoute` logs out on failed refresh in App effect; unauthorized users are redirected.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Enforced via backend auth and module enable settings; guards rely on auth/module settings to gate access.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- None in UI components; billing info shown in pages that consume billing store/endpoints.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- None; components react to store state and router changes.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Provider stack covers theme, metrics, toasts, error boundary, routing.
|
|
||||||
- Guards ensure unauthorized access is blocked at the route level.
|
|
||||||
- Global utilities avoid repeated boilerplate for scroll/error/loading behaviors.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Add new protected pages under `ProtectedRoute` and wrap with `ModuleGuard` when module-scoped.
|
|
||||||
- Use existing toast/error/loading utilities instead of duplicating.
|
|
||||||
- Keep provider order consistent (ErrorBoundary → Theme/Header/Toast → Router → App).
|
|
||||||
|
|
||||||
228
docs/30-FRONTEND/PAGES.md
Normal file
228
docs/30-FRONTEND/PAGES.md
Normal file
@@ -0,0 +1,228 @@
|
|||||||
|
# Frontend Pages & Routes
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Framework:** React 18 + TypeScript + React Router 6
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Route Configuration
|
||||||
|
|
||||||
|
Routes defined in `/frontend/src/App.tsx` with:
|
||||||
|
- Auth guards via `PrivateRoute` component
|
||||||
|
- Public routes for auth pages
|
||||||
|
- Nested layouts with sidebar
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Public Routes (No Auth Required)
|
||||||
|
|
||||||
|
| Route | Component | Description |
|
||||||
|
|-------|-----------|-------------|
|
||||||
|
| `/login` | `LoginPage` | User login |
|
||||||
|
| `/register` | `RegisterPage` | New account |
|
||||||
|
| `/forgot-password` | `ForgotPasswordPage` | Request reset |
|
||||||
|
| `/reset-password/:token` | `ResetPasswordPage` | Set new password |
|
||||||
|
| `/verify-email/:token` | `VerifyEmailPage` | Email verification |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Dashboard Routes
|
||||||
|
|
||||||
|
| Route | Component | Module | Description |
|
||||||
|
|-------|-----------|--------|-------------|
|
||||||
|
| `/` | `Dashboard` | - | Main dashboard |
|
||||||
|
| `/dashboard` | `Dashboard` | - | Main dashboard (alias) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planner Routes
|
||||||
|
|
||||||
|
| Route | Component | Module | Description |
|
||||||
|
|-------|-----------|--------|-------------|
|
||||||
|
| `/planner` | `PlannerPage` | Planner | Keyword management |
|
||||||
|
| `/planner/keywords` | `KeywordsPage` | Planner | Keywords list |
|
||||||
|
| `/planner/clusters` | `ClustersPage` | Planner | Cluster view |
|
||||||
|
| `/planner/ideas` | `ContentIdeasPage` | Planner | Content ideas |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Writer Routes
|
||||||
|
|
||||||
|
| Route | Component | Module | Description |
|
||||||
|
|-------|-----------|--------|-------------|
|
||||||
|
| `/writer` | `WriterPage` | Writer | Task management |
|
||||||
|
| `/writer/tasks` | `TasksPage` | Writer | Task list |
|
||||||
|
| `/writer/content` | `ContentListPage` | Writer | Content list |
|
||||||
|
| `/writer/content/:id` | `ContentEditorPage` | Writer | Edit content |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Automation Routes
|
||||||
|
|
||||||
|
| Route | Component | Module | Description |
|
||||||
|
|-------|-----------|--------|-------------|
|
||||||
|
| `/automation` | `AutomationPage` | Automation | Pipeline view |
|
||||||
|
| `/automation/config` | `AutomationConfigPage` | Automation | Configuration |
|
||||||
|
| `/automation/logs` | `AutomationLogsPage` | Automation | Activity logs |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Linker Routes
|
||||||
|
|
||||||
|
| Route | Component | Module | Description |
|
||||||
|
|-------|-----------|--------|-------------|
|
||||||
|
| `/linker` | `LinkerPage` | Linker | Internal linking (inactive) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Optimizer Routes
|
||||||
|
|
||||||
|
| Route | Component | Module | Description |
|
||||||
|
|-------|-----------|--------|-------------|
|
||||||
|
| `/optimizer` | `OptimizerPage` | Optimizer | SEO optimization (inactive) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Settings Routes
|
||||||
|
|
||||||
|
| Route | Component | Description |
|
||||||
|
|-------|-----------|-------------|
|
||||||
|
| `/settings` | `SettingsPage` | Settings index |
|
||||||
|
| `/settings/sites` | `SitesSettingsPage` | Site management |
|
||||||
|
| `/settings/sites/:id` | `SiteDetailPage` | Site details |
|
||||||
|
| `/settings/sectors` | `SectorsSettingsPage` | Sector management |
|
||||||
|
| `/settings/users` | `UsersSettingsPage` | User management |
|
||||||
|
| `/settings/integrations` | `IntegrationsSettingsPage` | Integration setup |
|
||||||
|
| `/settings/integrations/:id` | `IntegrationDetailPage` | Integration details |
|
||||||
|
| `/settings/prompts` | `PromptsSettingsPage` | AI prompts |
|
||||||
|
| `/settings/modules` | `ModulesSettingsPage` | Module enable/disable |
|
||||||
|
| `/settings/api-keys` | `APIKeysSettingsPage` | AI API keys |
|
||||||
|
| `/settings/billing` | `BillingSettingsPage` | Credit/billing |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Profile Routes
|
||||||
|
|
||||||
|
| Route | Component | Description |
|
||||||
|
|-------|-----------|-------------|
|
||||||
|
| `/profile` | `ProfilePage` | User profile |
|
||||||
|
| `/profile/account` | `AccountPage` | Account settings |
|
||||||
|
| `/profile/security` | `SecurityPage` | Password/2FA |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Page Components Location
|
||||||
|
|
||||||
|
```
|
||||||
|
frontend/src/pages/
|
||||||
|
├── auth/
|
||||||
|
│ ├── LoginPage.tsx
|
||||||
|
│ ├── RegisterPage.tsx
|
||||||
|
│ ├── ForgotPasswordPage.tsx
|
||||||
|
│ └── ResetPasswordPage.tsx
|
||||||
|
├── dashboard/
|
||||||
|
│ └── Dashboard.tsx
|
||||||
|
├── planner/
|
||||||
|
│ ├── PlannerPage.tsx
|
||||||
|
│ ├── KeywordsPage.tsx
|
||||||
|
│ ├── ClustersPage.tsx
|
||||||
|
│ └── ContentIdeasPage.tsx
|
||||||
|
├── writer/
|
||||||
|
│ ├── WriterPage.tsx
|
||||||
|
│ ├── TasksPage.tsx
|
||||||
|
│ ├── ContentListPage.tsx
|
||||||
|
│ └── ContentEditorPage.tsx
|
||||||
|
├── automation/
|
||||||
|
│ ├── AutomationPage.tsx
|
||||||
|
│ ├── AutomationConfigPage.tsx
|
||||||
|
│ └── AutomationLogsPage.tsx
|
||||||
|
├── linker/
|
||||||
|
│ └── LinkerPage.tsx
|
||||||
|
├── optimizer/
|
||||||
|
│ └── OptimizerPage.tsx
|
||||||
|
├── settings/
|
||||||
|
│ ├── SettingsPage.tsx
|
||||||
|
│ ├── SitesSettingsPage.tsx
|
||||||
|
│ ├── SectorsSettingsPage.tsx
|
||||||
|
│ ├── UsersSettingsPage.tsx
|
||||||
|
│ ├── IntegrationsSettingsPage.tsx
|
||||||
|
│ ├── PromptsSettingsPage.tsx
|
||||||
|
│ ├── ModulesSettingsPage.tsx
|
||||||
|
│ ├── APIKeysSettingsPage.tsx
|
||||||
|
│ └── BillingSettingsPage.tsx
|
||||||
|
└── profile/
|
||||||
|
├── ProfilePage.tsx
|
||||||
|
└── SecurityPage.tsx
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Route Guards
|
||||||
|
|
||||||
|
### PrivateRoute Component
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// frontend/src/components/auth/PrivateRoute.tsx
|
||||||
|
- Checks authentication state
|
||||||
|
- Redirects to /login if not authenticated
|
||||||
|
- Stores intended destination for post-login redirect
|
||||||
|
```
|
||||||
|
|
||||||
|
### Module-Based Visibility
|
||||||
|
|
||||||
|
Sidebar items hidden when module disabled:
|
||||||
|
- Controlled by `moduleStore.isModuleEnabled()`
|
||||||
|
- Does NOT block direct URL access (pending implementation)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Navigation Structure
|
||||||
|
|
||||||
|
### Main Sidebar (`AppSidebar.tsx`)
|
||||||
|
|
||||||
|
```
|
||||||
|
Dashboard
|
||||||
|
├── Planner [if enabled]
|
||||||
|
│ ├── Keywords
|
||||||
|
│ ├── Clusters
|
||||||
|
│ └── Ideas
|
||||||
|
├── Writer [if enabled]
|
||||||
|
│ ├── Tasks
|
||||||
|
│ └── Content
|
||||||
|
├── Automation [if enabled]
|
||||||
|
├── Linker [if enabled, hidden by default]
|
||||||
|
├── Optimizer [if enabled, hidden by default]
|
||||||
|
└── Settings
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Layout Components
|
||||||
|
|
||||||
|
| Component | Location | Purpose |
|
||||||
|
|-----------|----------|---------|
|
||||||
|
| `AppLayout` | `/layouts/AppLayout.tsx` | Main app wrapper |
|
||||||
|
| `AppSidebar` | `/components/sidebar/AppSidebar.tsx` | Navigation sidebar |
|
||||||
|
| `AppHeader` | `/components/header/AppHeader.tsx` | Top navigation |
|
||||||
|
| `PageHeader` | `/components/common/PageHeader.tsx` | Page title section |
|
||||||
|
| `ContentContainer` | `/components/common/ContentContainer.tsx` | Content wrapper |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## State Context
|
||||||
|
|
||||||
|
All routes have access to:
|
||||||
|
- `useAuthStore()` - User/account info
|
||||||
|
- `useSiteStore()` - Current site selection
|
||||||
|
- `useSectorStore()` - Current sector selection
|
||||||
|
- `useModuleStore()` - Module enable states
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
|
||||||
|
| Item | Description | Priority |
|
||||||
|
|------|-------------|----------|
|
||||||
|
| Route-level guards | Block disabled module routes, not just hide sidebar | High |
|
||||||
|
| Breadcrumbs | Add breadcrumb navigation | Medium |
|
||||||
|
| Route analytics | Track page views | Low |
|
||||||
@@ -1,90 +0,0 @@
|
|||||||
# State Management
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe how the frontend manages global state with Zustand stores, including authentication, site context, billing, and settings.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Auth store: `frontend/src/store/authStore.ts`
|
|
||||||
- Site store: `frontend/src/store/siteStore.ts`
|
|
||||||
- Billing store: `frontend/src/store/billingStore.ts`
|
|
||||||
- Settings store: `frontend/src/store/settingsStore.ts`
|
|
||||||
- Other UI/state stores: `frontend/src/store/onboardingStore.ts`, `columnVisibilityStore.ts`, `pageSizeStore.ts`, `settingsStore.ts`, etc.
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Persist authentication state (user, tokens), refresh sessions, and enforce account/plan presence.
|
|
||||||
- Track the active site (and sector via sector store) across modules, with persistence and change events.
|
|
||||||
- Fetch and cache billing balances/usage/limits.
|
|
||||||
- Load and update account/module settings with coalesced fetches and persistence.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- `authStore`:
|
|
||||||
- Persists `user`, `token`, `refreshToken`, `isAuthenticated`, `loading`.
|
|
||||||
- `login(email,password)`: POST `/api/v1/auth/login/`; stores tokens/user; errors mapped to PLAN_REQUIRED/ACCOUNT_REQUIRED/AUTH_FAILED; resets loading on error.
|
|
||||||
- `register(data)`: POST `/api/v1/auth/register/`; stores tokens/user; resets loading on error.
|
|
||||||
- `refreshToken()`: POST `/api/v1/auth/refresh/`; updates access token; attempts `refreshUser`; logs out on failure.
|
|
||||||
- `refreshUser()`: GET `/api/v1/auth/users/me/`; requires account+plan; updates user or logs out on auth failure; clears loading guard.
|
|
||||||
- `logout()`: clears all auth fields.
|
|
||||||
- `siteStore`:
|
|
||||||
- Persists `activeSite`; stores `loading`/`error`.
|
|
||||||
- `setActiveSite(site)`: sets/persists site, dispatches `siteChanged` event, triggers sector store load for the site.
|
|
||||||
- `loadActiveSite()`: fetches sites via `fetchSites`; picks prior site if still active, else first active; persists selection.
|
|
||||||
- `refreshActiveSite()`: refreshes current site from server; reloads if inactive/missing.
|
|
||||||
- `billingStore`:
|
|
||||||
- Holds `balance`, `usageSummary`, `usageLimits`, `loading`/`error`, `lastUpdated`.
|
|
||||||
- `loadBalance()`: calls `getCreditBalance`; keeps prior balance during retry; sets `lastUpdated`.
|
|
||||||
- `loadUsageSummary(startDate?, endDate?)`: calls `getCreditUsageSummary`; updates summary.
|
|
||||||
- `loadUsageLimits()`: calls `getCreditUsageLimits`; tolerates 404 by returning null; records errors otherwise.
|
|
||||||
- `reset()`: clears billing state.
|
|
||||||
- `settingsStore`:
|
|
||||||
- Persists `accountSettings`, `moduleSettings`, `moduleEnableSettings`, `loading`/`error`.
|
|
||||||
- Account settings: load all, load one, create/update with create/update endpoints; handles typed errors (not found vs validation).
|
|
||||||
- Module settings: load/update per module; stores per-module map.
|
|
||||||
- Module enable settings: caches for 60s, coalesces concurrent fetches; `isModuleEnabled` helper.
|
|
||||||
- `reset()` clears settings state.
|
|
||||||
- Other stores:
|
|
||||||
- `onboardingStore`, `columnVisibilityStore`, `pageSizeStore` manage UI/onboarding/table preferences (see respective files for exact fields/actions).
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Store state interfaces defined in each file (auth user shape, site, billing balance/usage, settings maps).
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Components/hooks read/write state via `useXStore` hooks.
|
|
||||||
- Auth flows call store actions, which perform fetches and update state; ProtectedRoute/ModuleGuard rely on auth and settings/module enable flags.
|
|
||||||
- Site changes dispatch `siteChanged` for downstream listeners and trigger sector loading.
|
|
||||||
- Billing/settings loads fetch data on demand and cache/persist where configured.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Auth tokens consumed by `fetchAPI` for all API calls.
|
|
||||||
- Active site influences planner/writer/automation calls via query params/context.
|
|
||||||
- Module enable settings influence `ModuleGuard`.
|
|
||||||
- Billing data feeds billing pages and credit displays.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Auth: unauthenticated → login/register → authenticated; refresh updates tokens/user; logout clears.
|
|
||||||
- Site: load/refresh/set updates `activeSite` and notifies listeners.
|
|
||||||
- Billing/settings: load/update actions mutate cached maps and loading/error flags.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Auth store resets loading on all error paths; throws descriptive errors.
|
|
||||||
- Billing store tolerates missing limits (404) and surfaces messages; balance retry preserves prior state.
|
|
||||||
- Settings store differentiates not-found vs validation errors; stores messages in `error`.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Auth store requires account+plan; site store filters on accessible sites; billing/settings calls use authenticated endpoints tied to the user’s account.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- Enforced via backend; billing store reads balances/usage/limits and does not mutate credits.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- None; all store actions are user-driven fetches with immediate state updates.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Zustand `persist` used for auth/site/settings to survive reloads.
|
|
||||||
- Guarded loading flags and error handling prevent stuck states.
|
|
||||||
- Site changes broadcast events for dependent components/stores.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Use existing store actions for auth/site/billing/settings instead of duplicating fetch logic.
|
|
||||||
- When adding new shared state, create a dedicated store in `src/store` and persist only needed slices.
|
|
||||||
- Keep error/reset semantics consistent (always clear loading on failure).
|
|
||||||
|
|
||||||
375
docs/30-FRONTEND/STORES.md
Normal file
375
docs/30-FRONTEND/STORES.md
Normal file
@@ -0,0 +1,375 @@
|
|||||||
|
# Zustand State Management
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
**Framework:** Zustand 4 with persist middleware
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Store Architecture
|
||||||
|
|
||||||
|
All stores in `/frontend/src/store/` use Zustand with TypeScript.
|
||||||
|
|
||||||
|
**Key Patterns:**
|
||||||
|
- `persist` middleware for localStorage persistence
|
||||||
|
- Async actions for API calls
|
||||||
|
- Selectors for derived state
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Auth Store (`authStore.ts`)
|
||||||
|
|
||||||
|
**Purpose:** User authentication and session management
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
interface AuthState {
|
||||||
|
user: User | null;
|
||||||
|
account: Account | null;
|
||||||
|
isAuthenticated: boolean;
|
||||||
|
accessToken: string | null;
|
||||||
|
refreshToken: string | null;
|
||||||
|
isLoading: boolean;
|
||||||
|
error: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface AuthActions {
|
||||||
|
login(email: string, password: string): Promise<void>;
|
||||||
|
logout(): void;
|
||||||
|
register(data: RegisterData): Promise<void>;
|
||||||
|
refreshAccessToken(): Promise<void>;
|
||||||
|
fetchUser(): Promise<void>;
|
||||||
|
updateUser(data: UserUpdate): Promise<void>;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Persistence:** `accessToken`, `refreshToken` in localStorage
|
||||||
|
|
||||||
|
**Usage:**
|
||||||
|
```typescript
|
||||||
|
const { user, login, logout, isAuthenticated } = useAuthStore();
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Site Store (`siteStore.ts`)
|
||||||
|
|
||||||
|
**Purpose:** Site selection and management
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
interface SiteState {
|
||||||
|
sites: Site[];
|
||||||
|
currentSite: Site | null;
|
||||||
|
isLoading: boolean;
|
||||||
|
error: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface SiteActions {
|
||||||
|
fetchSites(): Promise<void>;
|
||||||
|
createSite(data: SiteCreate): Promise<Site>;
|
||||||
|
updateSite(id: string, data: SiteUpdate): Promise<Site>;
|
||||||
|
deleteSite(id: string): Promise<void>;
|
||||||
|
setCurrentSite(site: Site): void;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Persistence:** `currentSite.id` in localStorage
|
||||||
|
|
||||||
|
**Auto-selection:** If no site selected and sites exist, auto-selects first site
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Sector Store (`sectorStore.ts`)
|
||||||
|
|
||||||
|
**Purpose:** Sector selection and management within sites
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
interface SectorState {
|
||||||
|
sectors: Sector[];
|
||||||
|
currentSector: Sector | null;
|
||||||
|
isLoading: boolean;
|
||||||
|
error: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface SectorActions {
|
||||||
|
fetchSectors(siteId: string): Promise<void>;
|
||||||
|
createSector(data: SectorCreate): Promise<Sector>;
|
||||||
|
updateSector(id: string, data: SectorUpdate): Promise<Sector>;
|
||||||
|
deleteSector(id: string): Promise<void>;
|
||||||
|
setCurrentSector(sector: Sector): void;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Persistence:** `currentSector.id` in localStorage
|
||||||
|
|
||||||
|
**Dependency:** Reloads when `currentSite` changes
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Module Store (`moduleStore.ts`)
|
||||||
|
|
||||||
|
**Purpose:** Track which modules are enabled/disabled
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
interface ModuleState {
|
||||||
|
modules: ModuleSettings;
|
||||||
|
isLoading: boolean;
|
||||||
|
error: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ModuleSettings {
|
||||||
|
planner_enabled: boolean;
|
||||||
|
writer_enabled: boolean;
|
||||||
|
linker_enabled: boolean;
|
||||||
|
optimizer_enabled: boolean;
|
||||||
|
automation_enabled: boolean;
|
||||||
|
integration_enabled: boolean;
|
||||||
|
publisher_enabled: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ModuleActions {
|
||||||
|
fetchModules(): Promise<void>;
|
||||||
|
updateModules(settings: Partial<ModuleSettings>): Promise<void>;
|
||||||
|
isModuleEnabled(module: ModuleName): boolean;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Usage:**
|
||||||
|
```typescript
|
||||||
|
const { isModuleEnabled } = useModuleStore();
|
||||||
|
if (isModuleEnabled('planner')) { /* show planner */ }
|
||||||
|
```
|
||||||
|
|
||||||
|
**Currently used for:** Sidebar visibility only
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Billing Store (`billingStore.ts`)
|
||||||
|
|
||||||
|
**Purpose:** Credit balance and usage tracking
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
interface BillingState {
|
||||||
|
balance: CreditBalance | null;
|
||||||
|
usage: CreditUsage[];
|
||||||
|
limits: PlanLimits | null;
|
||||||
|
isLoading: boolean;
|
||||||
|
error: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface CreditBalance {
|
||||||
|
ideaCredits: number;
|
||||||
|
contentCredits: number;
|
||||||
|
imageCredits: number;
|
||||||
|
optimizationCredits: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface BillingActions {
|
||||||
|
fetchBalance(): Promise<void>;
|
||||||
|
fetchUsage(period?: string): Promise<void>;
|
||||||
|
fetchLimits(): Promise<void>;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Refresh triggers:**
|
||||||
|
- After content generation
|
||||||
|
- After image generation
|
||||||
|
- After optimization (when implemented)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planner Store (`plannerStore.ts`)
|
||||||
|
|
||||||
|
**Purpose:** Keywords, clusters, and content ideas state
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
interface PlannerState {
|
||||||
|
keywords: Keyword[];
|
||||||
|
clusters: Cluster[];
|
||||||
|
ideas: ContentIdea[];
|
||||||
|
selectedKeywords: string[];
|
||||||
|
filters: KeywordFilters;
|
||||||
|
isLoading: boolean;
|
||||||
|
error: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface PlannerActions {
|
||||||
|
fetchKeywords(siteId: string, sectorId?: string): Promise<void>;
|
||||||
|
createKeyword(data: KeywordCreate): Promise<Keyword>;
|
||||||
|
bulkDeleteKeywords(ids: string[]): Promise<void>;
|
||||||
|
autoCluster(keywordIds: string[]): Promise<void>;
|
||||||
|
generateIdeas(clusterId: string): Promise<void>;
|
||||||
|
setFilters(filters: Partial<KeywordFilters>): void;
|
||||||
|
selectKeywords(ids: string[]): void;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Writer Store (`writerStore.ts`)
|
||||||
|
|
||||||
|
**Purpose:** Tasks and content management
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
interface WriterState {
|
||||||
|
tasks: Task[];
|
||||||
|
content: Content[];
|
||||||
|
currentContent: Content | null;
|
||||||
|
filters: TaskFilters;
|
||||||
|
isLoading: boolean;
|
||||||
|
isGenerating: boolean;
|
||||||
|
error: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface WriterActions {
|
||||||
|
fetchTasks(siteId: string, sectorId?: string): Promise<void>;
|
||||||
|
createTask(data: TaskCreate): Promise<Task>;
|
||||||
|
generateContent(taskId: string): Promise<Content>;
|
||||||
|
fetchContent(contentId: string): Promise<Content>;
|
||||||
|
updateContent(id: string, data: ContentUpdate): Promise<Content>;
|
||||||
|
generateImages(contentId: string): Promise<void>;
|
||||||
|
publishToWordPress(contentId: string): Promise<void>;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Generation state:** `isGenerating` tracks active AI operations
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Automation Store (`automationStore.ts`)
|
||||||
|
|
||||||
|
**Purpose:** Automation pipeline state and control
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
interface AutomationState {
|
||||||
|
config: AutomationConfig | null;
|
||||||
|
currentRun: AutomationRun | null;
|
||||||
|
pipeline: PipelineOverview | null;
|
||||||
|
history: AutomationRun[];
|
||||||
|
logs: AutomationLog[];
|
||||||
|
isLoading: boolean;
|
||||||
|
error: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface AutomationActions {
|
||||||
|
fetchConfig(siteId: string): Promise<void>;
|
||||||
|
updateConfig(data: ConfigUpdate): Promise<void>;
|
||||||
|
startRun(siteId: string): Promise<void>;
|
||||||
|
pauseRun(runId: string): Promise<void>;
|
||||||
|
resumeRun(runId: string): Promise<void>;
|
||||||
|
cancelRun(runId: string): Promise<void>;
|
||||||
|
fetchPipeline(siteId: string): Promise<void>;
|
||||||
|
fetchLogs(runId: string): Promise<void>;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Polling:** Active runs trigger status polling every 5 seconds
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Integration Store (`integrationStore.ts`)
|
||||||
|
|
||||||
|
**Purpose:** WordPress integration management
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
interface IntegrationState {
|
||||||
|
integrations: SiteIntegration[];
|
||||||
|
currentIntegration: SiteIntegration | null;
|
||||||
|
syncStatus: SyncStatus | null;
|
||||||
|
isLoading: boolean;
|
||||||
|
isSyncing: boolean;
|
||||||
|
error: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface IntegrationActions {
|
||||||
|
fetchIntegrations(siteId: string): Promise<void>;
|
||||||
|
createIntegration(data: IntegrationCreate): Promise<SiteIntegration>;
|
||||||
|
testConnection(id: string): Promise<TestResult>;
|
||||||
|
triggerSync(id: string): Promise<void>;
|
||||||
|
fetchSyncStatus(id: string): Promise<SyncStatus>;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## UI Store (`uiStore.ts`)
|
||||||
|
|
||||||
|
**Purpose:** UI state (sidebar, modals, notifications)
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
interface UIState {
|
||||||
|
sidebarOpen: boolean;
|
||||||
|
sidebarCollapsed: boolean;
|
||||||
|
theme: 'light' | 'dark' | 'system';
|
||||||
|
notifications: Notification[];
|
||||||
|
}
|
||||||
|
|
||||||
|
interface UIActions {
|
||||||
|
toggleSidebar(): void;
|
||||||
|
collapseSidebar(): void;
|
||||||
|
setTheme(theme: Theme): void;
|
||||||
|
addNotification(notification: Notification): void;
|
||||||
|
removeNotification(id: string): void;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Persistence:** `theme`, `sidebarCollapsed` in localStorage
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Store Dependencies
|
||||||
|
|
||||||
|
```
|
||||||
|
authStore
|
||||||
|
│
|
||||||
|
└── siteStore (loads after auth)
|
||||||
|
│
|
||||||
|
└── sectorStore (loads when site changes)
|
||||||
|
│
|
||||||
|
├── plannerStore (scoped to site/sector)
|
||||||
|
├── writerStore (scoped to site/sector)
|
||||||
|
├── automationStore (scoped to site)
|
||||||
|
└── integrationStore (scoped to site)
|
||||||
|
|
||||||
|
moduleStore (global, loads once per account)
|
||||||
|
billingStore (global, loads once per account)
|
||||||
|
uiStore (local only, no API)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Store Files Location
|
||||||
|
|
||||||
|
```
|
||||||
|
frontend/src/store/
|
||||||
|
├── authStore.ts
|
||||||
|
├── siteStore.ts
|
||||||
|
├── sectorStore.ts
|
||||||
|
├── moduleStore.ts
|
||||||
|
├── billingStore.ts
|
||||||
|
├── plannerStore.ts
|
||||||
|
├── writerStore.ts
|
||||||
|
├── automationStore.ts
|
||||||
|
├── integrationStore.ts
|
||||||
|
├── uiStore.ts
|
||||||
|
└── index.ts (re-exports)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Best Practices
|
||||||
|
|
||||||
|
1. **Always scope to site/sector** when fetching module data
|
||||||
|
2. **Check module enabled** before showing features
|
||||||
|
3. **Handle loading states** in UI components
|
||||||
|
4. **Clear store on logout** to prevent data leaks
|
||||||
|
5. **Use selectors** for derived/filtered data
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planned Changes
|
||||||
|
|
||||||
|
| Item | Description | Priority |
|
||||||
|
|------|-------------|----------|
|
||||||
|
| Add `linkerStore` | State for internal linking results | Medium |
|
||||||
|
| Add `optimizerStore` | State for optimization results | Medium |
|
||||||
|
| Better error typing | Typed error codes per store | Low |
|
||||||
|
| DevTools integration | Zustand devtools middleware | Low |
|
||||||
@@ -1,149 +0,0 @@
|
|||||||
# Automation Frontend Module
|
|
||||||
|
|
||||||
**Location:** `frontend/src/pages/automation/`
|
|
||||||
**Purpose:** UI components and pages for automation module
|
|
||||||
|
|
||||||
## Source Files
|
|
||||||
|
|
||||||
# Automation Page (AI Automation Pipeline Dashboard)
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Provide a site-scoped dashboard to configure, run, pause, resume, and monitor the 7-stage automation pipeline. Surfaces pipeline overview, current run status, metrics, history, configuration modal, and credit sufficiency checks.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Primary page: `frontend/src/pages/Automation/AutomationPage.tsx`
|
|
||||||
- Service: `frontend/src/services/automationService` (config, current run, estimate, pipeline overview, runNow, pause, resume, publishWithoutReview)
|
|
||||||
- Metrics sources: `frontend/src/services/api` (`fetchKeywords`, `fetchClusters`, `fetchContentIdeas`, `fetchTasks`, `fetchContent`, `fetchContentImages`)
|
|
||||||
- UI components: `frontend/src/components/Automation/{ActivityLog,ConfigModal,RunHistory,CurrentProcessingCard}`, `frontend/src/components/dashboard/EnhancedMetricCard`, `frontend/src/components/common/ComponentCard`, `frontend/src/components/common/PageMeta`, `frontend/src/components/common/DebugSiteSelector`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Load automation config, current run, credit estimate, and pipeline overview for the active site.
|
|
||||||
- Poll current run and pipeline status while running/paused; refresh metrics regularly during runs.
|
|
||||||
- Provide controls: Run Now, Pause, Resume, Save Config, Publish Without Review.
|
|
||||||
- Show per-stage cards, current processing card, run history, and activity log.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Site binding: requires `useSiteStore.activeSite`; without it, page shows a “select a site” message.
|
|
||||||
- Initial load (`loadData`): parallel calls to `getConfig`, `getCurrentRun`, `estimate`, `getPipelineOverview` plus low-level metrics (keywords/clusters/ideas/tasks/content/images counts) per site_id.
|
|
||||||
- Polling: 5s interval; if run status is `running`/`paused`, refresh run, pipeline, and metrics; otherwise refresh pipeline only.
|
|
||||||
- Metrics: counts for keywords (total/new/mapped), clusters (total/new/mapped), ideas (total/new/queued/completed), tasks (total), content (total/draft/review/published), images (total/pending).
|
|
||||||
- Stage cards: derived from `STAGE_CONFIG` array representing 7 pipeline stages including manual review gate.
|
|
||||||
- Actions:
|
|
||||||
- Run Now: checks credit sufficiency from `estimate`; blocks if insufficient.
|
|
||||||
- Pause/Resume: call automationService with site + run_id, then refresh run/pipeline/metrics.
|
|
||||||
- Save Config: persists partial config, updates local state, reloads pipeline/metrics/run.
|
|
||||||
- Publish Without Review: calls `publishWithoutReview` with confirmation prompt, then reloads.
|
|
||||||
- UI states: `loading` spinner until initial fetch; `showProcessingCard` toggled on when a run exists.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `AutomationConfig`: site-level automation settings (intervals, gates, etc.).
|
|
||||||
- `AutomationRun`: run_id, status (`running`, `paused`, etc.), stage info.
|
|
||||||
- `PipelineStage`: stage list with status/progress.
|
|
||||||
- Metrics DTOs from planner/writer/image endpoints (`count` fields only).
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- `useEffect` with site dependency → `loadData`.
|
|
||||||
- Interval polling while active runs → `loadCurrentRun`, `loadPipelineOverview`, `loadMetrics`.
|
|
||||||
- User actions dispatch to automationService; toasts for success/error; follow-up refreshes.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Pulls planner/writer/image counts to present authoritative pipeline context.
|
|
||||||
- Uses `useSiteStore` for tenant/site scoping; credit checks rely on billing estimate API.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- `currentRun.status` drives polling and control availability.
|
|
||||||
- `showProcessingCard` turns on when a run exists.
|
|
||||||
- `estimate.sufficient` gates Run Now action.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Toast-based error reporting; fetch failures log to console but keep page usable.
|
|
||||||
- Metrics fetch wrapped in try/catch; failure degrades to missing metrics without blocking the rest.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All automation service calls require `activeSite.id`; backend enforces account/site scoping. No client override for other tenants.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Run Now is blocked when `estimate.sufficient` is false; shows required vs current credits.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Client-side polling (5s) during active runs; no background scheduling beyond that.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Polling keeps UI aligned with long-running Celery pipeline states.
|
|
||||||
- Credit gate prevents user-initiated runs that would immediately fail server-side.
|
|
||||||
- Metrics are read-only mirrors of backend aggregates to align UI with planner/writer state.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- When adding new stages, extend `STAGE_CONFIG` and ensure backend pipeline overview includes them.
|
|
||||||
- Keep polling interval modest; if adding heavier metrics, consider staggering fetches to avoid rate limits.
|
|
||||||
- Wire new config fields through `AutomationConfig` type, `ConfigModal`, and `updateConfig` payload.
|
|
||||||
# Automation Components
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe the reusable UI components that compose the automation dashboard: stage cards, current processing card, run history, activity log, and configuration modal. These components visualize pipeline state, history, and settings sourced from automation services.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Stage cards & layout: `frontend/src/pages/Automation/AutomationPage.tsx` (renders stage cards from `STAGE_CONFIG`)
|
|
||||||
- Current run card: `frontend/src/components/Automation/CurrentProcessingCard.tsx`
|
|
||||||
- Activity log: `frontend/src/components/Automation/ActivityLog.tsx`
|
|
||||||
- Run history: `frontend/src/components/Automation/RunHistory.tsx`
|
|
||||||
- Config modal: `frontend/src/components/Automation/ConfigModal.tsx`
|
|
||||||
- Shared UI: `frontend/src/components/common/{ComponentCard,PageMeta,DebugSiteSelector}`, `frontend/src/components/dashboard/EnhancedMetricCard`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Stage cards: show each of the 7 pipeline stages with icon/color/status derived from pipeline overview.
|
|
||||||
- CurrentProcessingCard: surface active run details, stage name, status, percent, timestamps, and controls (Pause/Resume).
|
|
||||||
- ActivityLog: list recent automation events (from run log feed).
|
|
||||||
- RunHistory: show prior runs with status and timestamps.
|
|
||||||
- ConfigModal: edit and persist automation configuration per site.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Stage Cards:
|
|
||||||
- Built from `STAGE_CONFIG` array (keywords→clusters, clusters→ideas, ideas→tasks, tasks→content, content→image prompts, image prompts→images, manual review).
|
|
||||||
- Status/progress comes from `pipelineOverview.stages` provided by `automationService.getPipelineOverview`.
|
|
||||||
- CurrentProcessingCard:
|
|
||||||
- Receives `currentRun` and shows status; displays pause/resume buttons wired to page handlers that call `automationService.pause/resume`.
|
|
||||||
- Hidden when no current run; toggled by `showProcessingCard`.
|
|
||||||
- RunHistory:
|
|
||||||
- Takes run list (from `automationService.getCurrentRun` payload history) and renders chronological entries.
|
|
||||||
- ActivityLog:
|
|
||||||
- Displays textual log entries for the active run; consumes run log data supplied by the page.
|
|
||||||
- ConfigModal:
|
|
||||||
- Opens from page button; on save calls `automationService.updateConfig(activeSite.id, newConfig)`; merges into local config and refreshes pipeline/metrics.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `AutomationRun` (id, status, stage, progress, started_at/ended_at).
|
|
||||||
- `PipelineStage` array with stage identifiers, names, progress.
|
|
||||||
- `AutomationConfig` fields shown in modal (intervals/gates/etc., defined server-side).
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Page loads run + pipeline → passes data into stage cards, processing card, history, activity log.
|
|
||||||
- User opens ConfigModal → submit triggers updateConfig → page reloads pipeline/metrics/run to reflect new settings.
|
|
||||||
- Pause/Resume buttons on CurrentProcessingCard call page handlers, which in turn call automationService.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Components depend on site context from `useSiteStore` and data from automationService; no direct planner/writer calls (metrics happen in page).
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Components are pure renderers; state (visibility, selected config) managed by `AutomationPage`.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Errors in save/pause/resume are surfaced by the page via toasts; components render based on provided props.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All data passed in is already scoped to `activeSite`; components do not alter scoping.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- None inside components; Run Now credit gating handled at page level.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- None; updates driven by page polling interval.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Separation of concerns: components stay presentational; network calls remain in page.
|
|
||||||
- Stage cards use color/icon metadata for fast visual scanning of pipeline status.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Add new stages by extending `STAGE_CONFIG` and ensuring pipeline overview includes the new stage id/status.
|
|
||||||
- Extend ConfigModal fields in sync with backend `AutomationConfig`; persist via automationService.
|
|
||||||
- Keep CurrentProcessingCard controls minimal; any new action should call automationService and refresh run/pipeline afterward.
|
|
||||||
@@ -1,183 +0,0 @@
|
|||||||
# Writer Frontend Module
|
|
||||||
|
|
||||||
**Location:** `frontend/src/pages/writer/`
|
|
||||||
**Purpose:** UI components and pages for writer module
|
|
||||||
|
|
||||||
## Source Files
|
|
||||||
|
|
||||||
# Writer Main Page (Queue/Drafts/Images Navigation)
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Serve as the entry surface for writer workflows, routing users to task queue, drafts, images, review, and published content views. It relies on shared table templates and writer navigation tabs defined in the content/tasks/images pages.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Key pages under Writer:
|
|
||||||
- Tasks (queue): `frontend/src/pages/Writer/Tasks.tsx`
|
|
||||||
- Content drafts/list: `frontend/src/pages/Writer/Content.tsx`
|
|
||||||
- Images: `frontend/src/pages/Writer/Images.tsx`
|
|
||||||
- Content view: `frontend/src/pages/Writer/ContentView.tsx`
|
|
||||||
- Navigation tabs defined inside Tasks/Content/Images pages (`writerTabs`).
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Present writer-specific navigation and headers.
|
|
||||||
- Delegate to module pages that implement task creation, AI generation, content listing, and image management.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Writer pages use shared navigation tabs (`writerTabs`: Queue, Drafts, Images, Review, Published) rendered via `ModuleNavigationTabs`.
|
|
||||||
- Each page sets a header (`PageHeader`) with badge icon and binds to the tab component for consistent navigation.
|
|
||||||
- State (filters, pagination, selections) is managed within each page; there is no global writer-state container beyond shared stores (`useSectorStore`, `usePageSizeStore`).
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Task, Content, ContentImage DTOs from `frontend/src/services/api` (writer endpoints).
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- User enters writer area via route (e.g., `/writer/tasks` or `/writer/content`).
|
|
||||||
- Navigation tabs switch routes; each route mounts its page and fetches data (tasks/content/images).
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Sector/site scoping via `useSectorStore` and backend query params in API calls.
|
|
||||||
- Optimization and image generation actions route into optimizer or image generation APIs.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Per-page loading/filters; navigation changes unmount current page and mount target page.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Each page uses toasts for API errors; no shared error surface beyond per-page banners.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Backend filters by account/site/sector; pages pass sector context via filters or rely on server defaults.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- None on navigation; individual actions (AI generation) consume credits on backend.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- None at the navigation level.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Tabs keep writer UX consistent; each page owns its data loading to avoid cross-coupling.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- When adding a new writer view (e.g., “Outlines”), add a tab entry and a route in `App.tsx`, and implement the page with the same header/tab pattern.
|
|
||||||
# Image Editor Page (Images List & Generation)
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Manage generated images linked to writer content: list images grouped by content, filter/search, trigger generation, update statuses, and download/view images. Uses table template patterns similar to content/tasks.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Page: `frontend/src/pages/Writer/Images.tsx`
|
|
||||||
- API: `frontend/src/services/api` (`fetchContentImages`, `fetchImageGenerationSettings`, `generateImages`, `bulkUpdateImagesStatus`, `deleteContent`, `bulkDeleteContent`)
|
|
||||||
- Config: `frontend/src/config/pages/images.config` (columns/filters)
|
|
||||||
- UI components: `frontend/src/components/common/ImageQueueModal`, `frontend/src/components/common/SingleRecordStatusUpdateModal`, `frontend/src/components/common/PageHeader`, `frontend/src/components/navigation/ModuleNavigationTabs`, `frontend/src/components/ui/modal`
|
|
||||||
- Hooks: `frontend/src/hooks/useResourceDebug` (AI logs toggle), `frontend/src/hooks/useProgressModal` (via modal usage pattern)
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Fetch and render content-image groups with client-side filtering/search/sorting/pagination.
|
|
||||||
- Trigger image generation for selected content with queue modal and provider/model selection.
|
|
||||||
- Update image status in bulk and delete images/content.
|
|
||||||
- Provide AI function log visibility when resource debug is enabled.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Data load: `loadImages` calls `fetchContentImages({})`, applies client-side search (`content_title`) and status filter (`overall_status`), sorts (default `content_title`), paginates client-side (page size 10), and adds `id` field mirroring `content_id` for selection.
|
|
||||||
- Filters: search text; status dropdown; sort controls; pagination tracked in local state.
|
|
||||||
- Actions:
|
|
||||||
- Generate images: opens `ImageQueueModal`, builds queue items, calls `generateImages` with provider/model; tracks taskId/model/provider state; shows AI logs when resource debug enabled.
|
|
||||||
- Bulk status update: `bulkUpdateImagesStatus` on selected ids.
|
|
||||||
- Delete (single/bulk): uses `deleteContent`/`bulkDeleteContent`.
|
|
||||||
- Download/view: handled by row actions in config (template-driven).
|
|
||||||
- Navigation: writer tabs rendered via `ModuleNavigationTabs` (Queue/Drafts/Images/Review/Published).
|
|
||||||
- Resource debug: AI logs captured only when `useResourceDebug` returns true; `addAiLog` appends logs for visibility.
|
|
||||||
- Loading UX: `loading` + `showContent` gating; debounced search resets page as needed.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `ContentImagesGroup`: content_id, content_title, overall_status, images[].
|
|
||||||
- `ContentImage`: individual image entries with URL/status/prompt (from API).
|
|
||||||
- Generation settings: provider/model options from `fetchImageGenerationSettings`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- `useEffect` → `loadImages`.
|
|
||||||
- Filters/sort/page changes → recompute client-side subsets.
|
|
||||||
- Generate action → open modal → call `generateImages` → optionally log steps → reload images.
|
|
||||||
- Status update/delete actions → call API → reload.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Tied to writer content records; delete actions use writer content endpoints.
|
|
||||||
- AI generation leverages shared API endpoints that consume credits server-side.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- `loading`/`showContent` manage render timing; modal open states for queue/status/image viewer.
|
|
||||||
- `aiLogs` maintained only when debug is enabled.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Toast errors for load/generate/update/delete; generation errors also recorded in AI logs when enabled.
|
|
||||||
- Debounced search handles errors gracefully by keeping prior data until reload.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Backend enforces account/site/sector; client passes no explicit tenant fields beyond any default filters in API layer.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Image generation consumes credits on backend; page performs no credit gating.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- None; generation is user-triggered, and polling is not used here.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Client-side pagination used because API returns grouped images; keeps UI responsive without extra endpoints.
|
|
||||||
- Resource debug toggle avoids unnecessary log storage unless explicitly enabled.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- If server adds server-side pagination/filtering, remove client-side slicing and pass filters to API.
|
|
||||||
- Extend row actions by updating `images.config` with handlers wired to new behaviors.
|
|
||||||
- Keep generation flow in sync with backend provider/model options; surface credit estimates if backend exposes them.
|
|
||||||
# Content Editor Page
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Display a single content record with full details for review/read-only inspection. Acts as the per-record viewer for content generated or managed by the Writer module.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Page: `frontend/src/pages/Writer/ContentView.tsx`
|
|
||||||
- Template: `frontend/src/templates/ContentViewTemplate` (renders the actual layout/content fields)
|
|
||||||
- API: `frontend/src/services/api` (`fetchContentById`)
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Fetch a specific content item by id from route param and render it via `ContentViewTemplate`.
|
|
||||||
- Validate id parameter, handle not-found/error states, and provide back navigation to content list.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Route: `/writer/content/:id`.
|
|
||||||
- On mount: validates `id` is numeric; on invalid, shows toast and redirects to `/writer/content`.
|
|
||||||
- Fetches content via `fetchContentById(contentId)`; sets `content` state and clears `loading`.
|
|
||||||
- Errors: shows toast (`Failed to load content`) and leaves `content` null.
|
|
||||||
- Back action: `onBack` navigates to `/writer/content`.
|
|
||||||
- Page metadata: sets document title/description via `PageMeta`.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Content DTO from writer API (includes title, body/html, status, external_url, etc.; structure defined server-side).
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- `useEffect` → validate id → fetch content → update state → render template.
|
|
||||||
- Template receives `{ content, loading, onBack }`.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Navigation back to writer drafts list; no direct cross-module calls.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- `loading` toggles during fetch; `content` set on success; invalid id triggers navigation away.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Toast errors for missing/invalid id or fetch failure; console logs errors.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Backend enforces account/site/sector scoping; client only supplies id from route.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- None within this view; generation/updates handled elsewhere.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- None.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Strict id validation avoids bad requests.
|
|
||||||
- Keeps view read-only; editing handled elsewhere (not in this component).
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- If adding inline editing, extend `ContentViewTemplate` and add PATCH/PUT calls; keep id validation and error handling intact.
|
|
||||||
@@ -1,148 +0,0 @@
|
|||||||
# Automation Module (Code-Sourced, Dec 2025)
|
|
||||||
|
|
||||||
Single canonical reference for IGNY8 automation (backend, frontend, and runtime behavior). Replaces all prior automation docs in this folder.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1) What Automation Does
|
|
||||||
- Runs the 7-stage pipeline across Planner/Writer:
|
|
||||||
1) Keywords → Clusters (AI)
|
|
||||||
2) Clusters → Ideas (AI)
|
|
||||||
3) Ideas → Tasks (Local)
|
|
||||||
4) Tasks → Content (AI)
|
|
||||||
5) Content → Image Prompts (AI)
|
|
||||||
6) Image Prompts → Images (AI)
|
|
||||||
7) Manual Review Gate (Manual)
|
|
||||||
- Per-site, per-account isolation. One run at a time per site; guarded by cache lock `automation_lock_{site_id}`.
|
|
||||||
- Scheduling via Celery beat (`automation.check_scheduled_automations`); execution via Celery tasks (`run_automation_task`, `resume_automation_task` / `continue_automation_task`).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2) Backend API (behavior + payloads)
|
|
||||||
Base: `/api/v1/automation/` (auth required; site must belong to user’s account).
|
|
||||||
|
|
||||||
- `GET config?site_id=`: returns or creates config with enable flag, frequency (`daily|weekly|monthly`), scheduled_time, stage_1..6 batch sizes, delays (`within_stage_delay`, `between_stage_delay`), last_run_at, next_run_at.
|
|
||||||
- `PUT update_config?site_id=`: same fields as above, updates in-place.
|
|
||||||
- `POST run_now?site_id=`: starts a manual run; enqueues `run_automation_task`. Fails if a run is already active or lock exists.
|
|
||||||
- `GET current_run?site_id=`: current running/paused run with status, current_stage, totals, and stage_1..7_result blobs (counts, credits, partial flags, skip reasons).
|
|
||||||
- `GET pipeline_overview?site_id=`: per-stage status counts and “pending” numbers for UI cards.
|
|
||||||
- `GET current_processing?site_id=&run_id=`: live processing snapshot for an active run; null if not running.
|
|
||||||
- `POST pause|resume|cancel?site_id=&run_id=`: pause after current item; resume from saved `current_stage`; cancel after current item and stamp cancelled_at/completed_at.
|
|
||||||
- `GET history?site_id=`: last 20 runs (id, status, trigger, timestamps, total_credits_used, current_stage).
|
|
||||||
- `GET logs?run_id=&lines=100`: tail of the per-run activity log written by AutomationLogger.
|
|
||||||
- `GET estimate?site_id=`: estimated_credits, current_balance, sufficient (balance >= 1.2x estimate).
|
|
||||||
|
|
||||||
Error behaviors:
|
|
||||||
- Missing site_id/run_id → 400.
|
|
||||||
- Site not in account → 404.
|
|
||||||
- Run not found → 404 on run-specific endpoints.
|
|
||||||
- Already running / lock held → 400 on run_now.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3) Data Model (runtime state)
|
|
||||||
- `AutomationConfig` (one per site): enable flag, schedule (frequency, time), batch sizes per stage (1–6), delays (within-stage, between-stage), last_run_at, next_run_at.
|
|
||||||
- `AutomationRun`: run_id, trigger_type (manual/scheduled), status (running/paused/cancelled/completed/failed), current_stage, timestamps (start/pause/resume/cancel/complete), total_credits_used, per-stage result JSON (stage_1_result … stage_7_result), error_message.
|
|
||||||
- Activity logs: one file per run via AutomationLogger; streamed through the `logs` endpoint.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4) How Execution Works (AutomationService)
|
|
||||||
- Start: grabs cache lock `automation_lock_{site_id}`, estimates credits, enforces 1.2x balance check, creates AutomationRun and log file.
|
|
||||||
- AI functions used: Stage 1 `AutoClusterFunction`; Stage 2 `GenerateIdeasFunction`; Stage 4 `GenerateContentFunction`; Stage 5 `GenerateImagePromptsFunction`; Stage 6 uses `process_image_generation_queue` (not the partial `generate_images` AI function).
|
|
||||||
- Stage flow (per code):
|
|
||||||
- Stage 1 Keywords → Clusters: require ≥5 keywords (validate_minimum_keywords); batch by config; AIEngine clustering; records keywords_processed, clusters_created, batches, credits, time; skips if insufficient keywords.
|
|
||||||
- Stage 2 Clusters → Ideas: batch by config; AIEngine ideas; records ideas_created.
|
|
||||||
- Stage 3 Ideas → Tasks: local conversion of queued ideas to tasks; batches by config; no AI.
|
|
||||||
- Stage 4 Tasks → Content: batch by config; AIEngine content; records content count + word totals.
|
|
||||||
- Stage 5 Content → Image Prompts: batch by config; AIEngine image-prompts into Images (featured + in-article).
|
|
||||||
- Stage 6 Image Prompts → Images: uses `process_image_generation_queue` with provider/model from IntegrationSettings; updates Images status.
|
|
||||||
- Stage 7 Manual Review Gate: marks ready-for-review counts; no AI.
|
|
||||||
- Control: each stage checks `_check_should_stop` (paused/cancelled); saves partial progress (counts, credits) before returning; resume continues from `current_stage`.
|
|
||||||
- Credits: upfront estimate check (1.2x buffer) before starting; AIEngine per-call pre-checks and post-SAVE deductions; `total_credits_used` accumulates.
|
|
||||||
- Locks: acquired on start; cleared on completion or failure; also cleared on fatal errors in tasks.
|
|
||||||
- Errors: any unhandled exception marks run failed, sets error_message, logs error, clears lock; pipeline_overview/history reflect status.
|
|
||||||
- Stage result fields (persisted):
|
|
||||||
- S1: keywords_processed, clusters_created, batches_run, credits_used, skipped/partial flags, time_elapsed.
|
|
||||||
- S2: clusters_processed, ideas_created, batches_run, credits_used.
|
|
||||||
- S3: ideas_processed, tasks_created, batches_run.
|
|
||||||
- S4: tasks_processed, content_created, total_words, batches_run, credits_used.
|
|
||||||
- S5: content_processed, prompts_created, batches_run, credits_used.
|
|
||||||
- S6: images_processed, images_generated, batches_run.
|
|
||||||
- S7: ready_for_review counts.
|
|
||||||
|
|
||||||
Batching & delays:
|
|
||||||
- Configurable per site; stage_1..6 batch sizes control how many items per batch; `within_stage_delay` pauses between batches; `between_stage_delay` between stages.
|
|
||||||
|
|
||||||
Scheduling:
|
|
||||||
- `check_scheduled_automations` runs hourly; respects frequency/time and last_run_at (~23h guard); skips if a run is active; sets next_run_at; starts `run_automation_task`.
|
|
||||||
|
|
||||||
Celery execution:
|
|
||||||
- `run_automation_task` runs stages 1→7 sequentially for a run_id; failures mark run failed and clear lock.
|
|
||||||
- `resume_automation_task` / `continue_automation_task` continue from saved `current_stage`.
|
|
||||||
- Workers need access to cache (locks) and IntegrationSettings (models/providers).
|
|
||||||
|
|
||||||
Image pipeline specifics:
|
|
||||||
- Stage 5 writes prompts to Images (featured + ordered in-article).
|
|
||||||
- Stage 6 generates images via queue helper; AI `generate_images` remains partial/broken and is not used by automation.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5) Frontend Behavior (AutomationPage)
|
|
||||||
- Route: `/automation`.
|
|
||||||
- What the user can do: run now, pause, resume, cancel; edit config (enable/schedule, batch sizes, delays); view activity log; view history; watch live processing card and pipeline cards update.
|
|
||||||
- Polling: every ~5s while a run is running/paused for current_run, pipeline_overview, metrics, current_processing; lighter polling when idle.
|
|
||||||
- Metrics: fetched via low-level endpoints (keywords/clusters/ideas/tasks/content/images) for authoritative counts.
|
|
||||||
- States shown: running, paused, cancelled, failed, completed; processing card shown when a run exists; pipeline cards use “pending” counts from pipeline_overview.
|
|
||||||
- Activity log: pulled from `logs` endpoint; shown in UI for live tailing.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 6) Configuration & Dependencies
|
|
||||||
- Needs IntegrationSettings for AI models and image providers (OpenAI/runware).
|
|
||||||
- Requires Celery beat and workers; cache backend required for locks.
|
|
||||||
- Tenant scoping everywhere: site + account filtering on all automation queries.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 7) Known Limitations and Gaps
|
|
||||||
- `generate_images` AI function is partial/broken; automation uses queue helper instead.
|
|
||||||
- Pause/Cancel stop after the current item; no mid-item abort.
|
|
||||||
- Batch defaults are conservative (e.g., stage_2=1, stage_4=1); tune per site for throughput.
|
|
||||||
- Stage 7 is manual; no automated review step.
|
|
||||||
- No automated test suite observed for automation pipeline (stage transitions, pause/resume/cancel, scheduling guards, credit estimation/deduction).
|
|
||||||
- Enhancements to consider: fix or replace `generate_images`; add mid-item abort; surface lock status/owner; broaden batch defaults after validation; add operator-facing doc in app; add tests.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 8) Field/Behavior Quick Tables
|
|
||||||
|
|
||||||
### Pipeline “pending” definitions (pipeline_overview)
|
|
||||||
- Stage 1: Keywords with status `new`, cluster is null, not disabled.
|
|
||||||
- Stage 2: Clusters status `new`, not disabled, with no ideas.
|
|
||||||
- Stage 3: ContentIdeas status `new`.
|
|
||||||
- Stage 4: Tasks status `queued`.
|
|
||||||
- Stage 5: Content status `draft` with zero images.
|
|
||||||
- Stage 6: Images status `pending`.
|
|
||||||
- Stage 7: Content status `review`.
|
|
||||||
|
|
||||||
### Stage result fields (stored on AutomationRun)
|
|
||||||
- S1: keywords_processed, clusters_created, batches_run, credits_used, skipped, partial, time_elapsed.
|
|
||||||
- S2: clusters_processed, ideas_created, batches_run, credits_used.
|
|
||||||
- S3: ideas_processed, tasks_created, batches_run.
|
|
||||||
- S4: tasks_processed, content_created, total_words, batches_run, credits_used.
|
|
||||||
- S5: content_processed, prompts_created, batches_run, credits_used.
|
|
||||||
- S6: images_processed, images_generated, batches_run.
|
|
||||||
- S7: ready_for_review.
|
|
||||||
|
|
||||||
### Credit handling
|
|
||||||
- Pre-run: estimate_credits * 1.2 vs account.credits (fails if insufficient).
|
|
||||||
- Per AI call: AIEngine pre-check credits; post-SAVE deduction with cost/tokens tracked; total_credits_used aggregates deductions.
|
|
||||||
|
|
||||||
### Logging
|
|
||||||
- Per-run log file via AutomationLogger; accessed with `GET logs?run_id=&lines=`; includes stage start/progress/errors and batch info.
|
|
||||||
|
|
||||||
### Polling (frontend)
|
|
||||||
- Active run: ~5s cadence for current_run, pipeline_overview, metrics, current_processing, logs tail.
|
|
||||||
- Idle: lighter polling (current_run/pipeline_overview) to show readiness and pending counts.
|
|
||||||
|
|
||||||
@@ -1,73 +0,0 @@
|
|||||||
# Content Lifecycle
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Trace content from keyword intake through clustering, idea generation, task creation, AI writing, image generation, and publishing/sync, mapping to backend services and frontend pages.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Planner models: `backend/igny8_core/business/planning/models.py` (`Keywords`, `Clusters`, `ContentIdeas`)
|
|
||||||
- Writer models: `backend/igny8_core/business/content/models.py` (`Tasks`, `Content`, `Images`, `ContentTaxonomy`)
|
|
||||||
- ViewSets: `backend/igny8_core/modules/planner/views.py`; `backend/igny8_core/modules/writer/views.py`
|
|
||||||
- Automation orchestration: `backend/igny8_core/business/automation/services/automation_service.py`; Celery tasks `business/automation/tasks.py`
|
|
||||||
- WordPress sync/publish: `backend/igny8_core/business/integration/services/integration_service.py`, `modules/integration/views.py`
|
|
||||||
- Frontend pages: `frontend/src/pages/Planner/*`, `frontend/src/pages/Writer/*`, `frontend/src/pages/Automation/AutomationPage.tsx`, `frontend/src/pages/Sites/*`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Collect and cluster keywords, generate ideas, create tasks, generate content and images, then publish/sync.
|
|
||||||
- Support manual steps via Planner/Writer pages and automated runs via Automation pipeline.
|
|
||||||
- Maintain site/sector scoping and credit enforcement throughout AI operations.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Keywords → Clusters: `KeywordViewSet` (upload/create) and `ClusterViewSet` (manual or AI auto_cluster). Models inherit `SiteSectorBaseModel` for scoping.
|
|
||||||
- Clusters → ContentIdeas: `ContentIdeasViewSet` with AI generation endpoint; ideas track status.
|
|
||||||
- Ideas/Tasks: `Tasks` model holds brief/keywords/structure/status; created manually or by automation stage.
|
|
||||||
- Tasks → Content: Writer endpoints generate content (AI) and update `Content` records; statuses managed in writer views.
|
|
||||||
- Content → Images: `ImagesViewSet` handles image generation and storage; images linked to tasks/content.
|
|
||||||
- Publishing: Integration service sends content to WordPress via `/api/v1/integration` endpoints; WP plugin responds with IDs and syncs status back.
|
|
||||||
- Automation: `automation_service.run_automation` executes 7 stages with delays/retries/credit estimates; run status tracked in `AutomationRun`.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Planner: `Keywords`, `Clusters`, `ContentIdeas`.
|
|
||||||
- Writer: `Tasks`, `Content`, `Images`, `ContentTaxonomy`.
|
|
||||||
- Automation: `AutomationConfig`, `AutomationRun`.
|
|
||||||
- Integration: `SiteIntegration`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Manual path: upload keywords → cluster (auto/manual) → generate ideas → create tasks → generate content → generate images → publish to WP → WP syncs status back.
|
|
||||||
- Automated path: Automation pipeline stages perform the same transitions in sequence, logging progress and deducting credits.
|
|
||||||
- Each stage uses DRF viewsets; automation uses Celery tasks (`run_automation_task`, `resume_automation_task`) and logger for trace files.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Automation consumes planner/writer endpoints and triggers integration publish.
|
|
||||||
- Integration relies on site’s WP API key (`Site.wp_api_key`) and `SiteIntegration` config for credentials/URLs.
|
|
||||||
- Billing deducts credits for AI operations through `CreditService`.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Keywords: status (active/inactive/used) per model.
|
|
||||||
- Clusters: status (active/archived).
|
|
||||||
- ContentIdeas: status (draft/approved/in_progress/completed).
|
|
||||||
- Tasks: status simplified (queued/completed) in Stage 1; content status (draft/published) and image status stored on related models.
|
|
||||||
- AutomationRun: status across run lifecycle (created/running/paused/completed/failed).
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Viewsets rely on unified DRF error responses; automation logger records failures per stage; Celery retries configured where applicable.
|
|
||||||
- WordPress sync errors surfaced via integration service responses.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All planner/writer/automation/integration models inherit account/site/sector via base models; viewsets filter by `request.account` and query params.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- AI clustering/idea/content/image generation deduct credits via `CreditService` cost map; automation blocks run if estimated credits insufficient.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Automation scheduler: `check_scheduled_automations` enqueues runs; `run_automation_task`/`resume_automation_task` execute pipeline in Celery.
|
|
||||||
- AI tasks run async via Celery wrappers (`ai/tasks.py`).
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Single source of truth for tenancy via base models; prevents cross-tenant data exposure.
|
|
||||||
- Automation mirrors manual flow; manual pages remain usable when automation is paused or disabled.
|
|
||||||
- WordPress publish/sync uses API key auth to avoid storing user credentials beyond site integration config.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- When adding new content stages, update both manual endpoints and automation stages.
|
|
||||||
- Ensure every new action that calls AI is wired through `CreditService` and respects site/sector filters.
|
|
||||||
- For new publishing targets, extend `SiteIntegration` and integration service while keeping site/account scoping.
|
|
||||||
337
docs/40-WORKFLOWS/CONTENT-PIPELINE.md
Normal file
337
docs/40-WORKFLOWS/CONTENT-PIPELINE.md
Normal file
@@ -0,0 +1,337 @@
|
|||||||
|
# Content Pipeline Workflow
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The IGNY8 content pipeline transforms raw keywords into published WordPress articles through a multi-stage workflow. This can run manually (step-by-step) or automatically via the Automation module.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Pipeline Stages
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────────────────┐
|
||||||
|
│ CONTENT PIPELINE │
|
||||||
|
├─────────────────────────────────────────────────────────────────────────┤
|
||||||
|
│ │
|
||||||
|
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||||
|
│ │ KEYWORDS │───►│ CLUSTERS │───►│ IDEAS │───►│ TASKS │ │
|
||||||
|
│ │ Stage 1 │ │ Stage 2 │ │ Stage 3 │ │ Stage 4 │ │
|
||||||
|
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
|
||||||
|
│ │
|
||||||
|
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||||
|
│ │ CONTENT │───►│ IMAGES │───►│ REVIEW │───►│ PUBLISH │ │
|
||||||
|
│ │ Stage 5 │ │ Stage 6 │ │ Stage 7 │ │ Stage 8 │ │
|
||||||
|
│ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │
|
||||||
|
│ │
|
||||||
|
└─────────────────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Stage 1: Keywords
|
||||||
|
|
||||||
|
**Module:** Planner
|
||||||
|
**Status Values:** `pending`, `clustered`, `used`, `archived`
|
||||||
|
|
||||||
|
### Input
|
||||||
|
- Seed keywords (manually added or from SEO tools)
|
||||||
|
- Optional: search volume, difficulty, CPC data
|
||||||
|
|
||||||
|
### Process
|
||||||
|
1. User adds keywords via UI or bulk import
|
||||||
|
2. Keywords validated and deduplicated
|
||||||
|
3. Assigned to site + sector
|
||||||
|
|
||||||
|
### Output
|
||||||
|
- Keyword records in `Keyword` model
|
||||||
|
- Status: `pending`
|
||||||
|
|
||||||
|
### Credit Usage
|
||||||
|
- None (free operation)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Stage 2: Clustering
|
||||||
|
|
||||||
|
**Module:** Planner
|
||||||
|
**AI Function:** `AutoClusterKeywords`
|
||||||
|
|
||||||
|
### Input
|
||||||
|
- Selected pending keywords (2-100)
|
||||||
|
|
||||||
|
### Process
|
||||||
|
1. AI analyzes semantic relationships
|
||||||
|
2. Groups keywords by topic/intent
|
||||||
|
3. Creates cluster with name + description
|
||||||
|
|
||||||
|
### Output
|
||||||
|
- `Cluster` records created
|
||||||
|
- Keywords linked to clusters
|
||||||
|
- Keyword status → `clustered`
|
||||||
|
|
||||||
|
### Credit Usage
|
||||||
|
- 1 idea credit per clustering operation
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Stage 3: Ideas
|
||||||
|
|
||||||
|
**Module:** Planner
|
||||||
|
**AI Function:** `GenerateContentIdeas`
|
||||||
|
|
||||||
|
### Input
|
||||||
|
- Cluster with 2+ keywords
|
||||||
|
|
||||||
|
### Process
|
||||||
|
1. AI generates content idea titles
|
||||||
|
2. Creates brief description for each
|
||||||
|
3. Suggests primary + secondary keywords
|
||||||
|
|
||||||
|
### Output
|
||||||
|
- `ContentIdea` records created
|
||||||
|
- Status: `pending`
|
||||||
|
|
||||||
|
### Credit Usage
|
||||||
|
- 1 idea credit per idea generated
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Stage 4: Tasks
|
||||||
|
|
||||||
|
**Module:** Writer
|
||||||
|
**Status Values:** `pending`, `in_progress`, `completed`, `cancelled`
|
||||||
|
|
||||||
|
### Input
|
||||||
|
- Selected content ideas
|
||||||
|
|
||||||
|
### Process
|
||||||
|
1. Ideas converted to tasks
|
||||||
|
2. Task gets content brief + keywords
|
||||||
|
3. Optional: set due date, assign user
|
||||||
|
|
||||||
|
### Output
|
||||||
|
- `Task` records created
|
||||||
|
- Status: `pending`
|
||||||
|
- Ideas status → `used`
|
||||||
|
|
||||||
|
### Credit Usage
|
||||||
|
- None (free operation)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Stage 5: Content Generation
|
||||||
|
|
||||||
|
**Module:** Writer
|
||||||
|
**AI Function:** `GenerateContent`
|
||||||
|
|
||||||
|
### Input
|
||||||
|
- Task with title + keywords + brief
|
||||||
|
|
||||||
|
### Process
|
||||||
|
1. AI generates full article content
|
||||||
|
2. Creates structured HTML output
|
||||||
|
3. Adds meta title + description
|
||||||
|
|
||||||
|
### Output
|
||||||
|
- `Content` record created
|
||||||
|
- Full HTML body
|
||||||
|
- SEO metadata
|
||||||
|
- Task status → `completed`
|
||||||
|
|
||||||
|
### Credit Usage
|
||||||
|
- 1 content credit per generation
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Stage 6: Image Generation
|
||||||
|
|
||||||
|
**Module:** Writer
|
||||||
|
**AI Function:** `GenerateImages`
|
||||||
|
|
||||||
|
### Input
|
||||||
|
- Content record
|
||||||
|
- Number of images (default: 1-3)
|
||||||
|
|
||||||
|
### Process
|
||||||
|
1. AI analyzes content for image prompts
|
||||||
|
2. Generates images via DALL-E/Runware
|
||||||
|
3. Creates thumbnail + full versions
|
||||||
|
|
||||||
|
### Output
|
||||||
|
- `ContentImage` records created
|
||||||
|
- Image URLs + alt text
|
||||||
|
- Featured image assigned
|
||||||
|
|
||||||
|
### Credit Usage
|
||||||
|
- 1 image credit per image generated
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Stage 7: Review
|
||||||
|
|
||||||
|
**Module:** Writer
|
||||||
|
**Status Values:** `draft`, `review`, `approved`, `published`
|
||||||
|
|
||||||
|
### Input
|
||||||
|
- Generated content + images
|
||||||
|
|
||||||
|
### Process
|
||||||
|
1. Content displayed in rich editor
|
||||||
|
2. User reviews + edits if needed
|
||||||
|
3. User approves for publishing
|
||||||
|
|
||||||
|
### Output
|
||||||
|
- Content status → `approved`
|
||||||
|
- Any manual edits saved
|
||||||
|
|
||||||
|
### Credit Usage
|
||||||
|
- None (free operation)
|
||||||
|
- Regeneration costs additional credits
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Stage 8: Publishing
|
||||||
|
|
||||||
|
**Module:** Publisher
|
||||||
|
**Integration:** WordPress REST API
|
||||||
|
|
||||||
|
### Input
|
||||||
|
- Approved content
|
||||||
|
- WordPress integration credentials
|
||||||
|
|
||||||
|
### Process
|
||||||
|
1. Content formatted for WordPress
|
||||||
|
2. Images uploaded to WP media
|
||||||
|
3. Post created with categories/tags
|
||||||
|
4. Status set to draft/published
|
||||||
|
|
||||||
|
### Output
|
||||||
|
- `PublishingRecord` created
|
||||||
|
- WordPress post ID stored
|
||||||
|
- Content status → `published`
|
||||||
|
|
||||||
|
### Credit Usage
|
||||||
|
- None (free operation)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Automation Mode
|
||||||
|
|
||||||
|
When running via Automation module:
|
||||||
|
|
||||||
|
1. **Configuration** - Set limits per stage
|
||||||
|
2. **Execution** - Pipeline runs automatically
|
||||||
|
3. **Pacing** - Configurable delays between operations
|
||||||
|
4. **Monitoring** - Real-time status updates
|
||||||
|
|
||||||
|
### Automation Config Options
|
||||||
|
|
||||||
|
```
|
||||||
|
Stage Limits:
|
||||||
|
- clustering_limit: Max keywords to cluster
|
||||||
|
- ideas_limit: Max ideas to generate
|
||||||
|
- content_limit: Max content to generate
|
||||||
|
- image_limit: Max images to generate
|
||||||
|
- publish_limit: Max content to publish
|
||||||
|
|
||||||
|
Timing:
|
||||||
|
- delay_between_operations: Seconds between API calls
|
||||||
|
- max_runtime: Maximum run duration
|
||||||
|
|
||||||
|
Behavior:
|
||||||
|
- auto_approve: Skip review stage
|
||||||
|
- auto_publish: Publish immediately
|
||||||
|
- stop_on_error: Halt pipeline on failure
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Data Flow Diagram
|
||||||
|
|
||||||
|
```
|
||||||
|
Seed Keywords
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
┌─────────────────┐
|
||||||
|
│ Keyword │ status: pending
|
||||||
|
│ Model │ belongs_to: site, sector
|
||||||
|
└────────┬────────┘
|
||||||
|
│ AI: AutoCluster
|
||||||
|
▼
|
||||||
|
┌─────────────────┐
|
||||||
|
│ Cluster │ keywords: [...]
|
||||||
|
│ Model │ belongs_to: site, sector
|
||||||
|
└────────┬────────┘
|
||||||
|
│ AI: GenerateIdeas
|
||||||
|
▼
|
||||||
|
┌─────────────────┐
|
||||||
|
│ ContentIdea │ primary_keyword, secondaries
|
||||||
|
│ Model │ cluster_id, status
|
||||||
|
└────────┬────────┘
|
||||||
|
│ Convert to Task
|
||||||
|
▼
|
||||||
|
┌─────────────────┐
|
||||||
|
│ Task │ idea_id, brief
|
||||||
|
│ Model │ assigned_to, status
|
||||||
|
└────────┬────────┘
|
||||||
|
│ AI: GenerateContent
|
||||||
|
▼
|
||||||
|
┌─────────────────┐
|
||||||
|
│ Content │ task_id, body, meta
|
||||||
|
│ Model │ status: draft
|
||||||
|
└────────┬────────┘
|
||||||
|
│ AI: GenerateImages
|
||||||
|
▼
|
||||||
|
┌─────────────────┐
|
||||||
|
│ ContentImage │ content_id, url
|
||||||
|
│ Model │ alt_text, is_featured
|
||||||
|
└────────┬────────┘
|
||||||
|
│ User Review
|
||||||
|
▼
|
||||||
|
┌─────────────────┐
|
||||||
|
│ Content │ status: approved
|
||||||
|
│ (updated) │
|
||||||
|
└────────┬────────┘
|
||||||
|
│ WordPress API
|
||||||
|
▼
|
||||||
|
┌─────────────────┐
|
||||||
|
│PublishingRecord │ content_id, wp_post_id
|
||||||
|
│ Model │ status, published_at
|
||||||
|
└─────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
|
||||||
|
| Stage | Common Errors | Recovery |
|
||||||
|
|-------|---------------|----------|
|
||||||
|
| Clustering | API timeout | Retry with smaller batch |
|
||||||
|
| Ideas | API rate limit | Wait and retry |
|
||||||
|
| Content | Insufficient credits | Add credits, retry |
|
||||||
|
| Images | Image API failure | Skip images, continue |
|
||||||
|
| Publish | WordPress auth fail | Reauth integration |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Monitoring
|
||||||
|
|
||||||
|
### Pipeline Stats (Dashboard)
|
||||||
|
|
||||||
|
- Keywords pending clustering
|
||||||
|
- Ideas pending task creation
|
||||||
|
- Tasks pending generation
|
||||||
|
- Content pending review
|
||||||
|
- Content pending publish
|
||||||
|
|
||||||
|
### Automation Logs
|
||||||
|
|
||||||
|
- Run ID + timestamps
|
||||||
|
- Stage + item processed
|
||||||
|
- Success/failure status
|
||||||
|
- Credit deductions
|
||||||
|
- Error messages
|
||||||
399
docs/40-WORKFLOWS/CREDIT-SYSTEM.md
Normal file
399
docs/40-WORKFLOWS/CREDIT-SYSTEM.md
Normal file
@@ -0,0 +1,399 @@
|
|||||||
|
# Usage & Content System
|
||||||
|
|
||||||
|
**Last Verified:** December 25, 2025
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
IGNY8 uses a content-based allowance system. Users see "Content Pieces" while the backend tracks detailed credit consumption for internal cost monitoring.
|
||||||
|
|
||||||
|
**User View:** `47/50 Content Pieces Remaining`
|
||||||
|
**Backend Tracks:** Idea credits, content credits, image credits (for cost analysis)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## How It Works
|
||||||
|
|
||||||
|
### User-Facing (Simple)
|
||||||
|
|
||||||
|
| What Users See | Description |
|
||||||
|
|----------------|-------------|
|
||||||
|
| **Content Pieces** | Monthly allowance of pages/articles |
|
||||||
|
| **X/Y Remaining** | Used vs total for the month |
|
||||||
|
| **Upgrade Plan** | Get more content pieces |
|
||||||
|
|
||||||
|
### Backend (Detailed - Internal Only)
|
||||||
|
|
||||||
|
| Credit Type | Used For | Tracked For |
|
||||||
|
|-------------|----------|-------------|
|
||||||
|
| Idea Credits | Clustering, idea generation | Cost analysis |
|
||||||
|
| Content Credits | Article generation | Usage limits |
|
||||||
|
| Image Credits | Image generation | Cost analysis |
|
||||||
|
| Optimization Credits | SEO optimization (future) | Cost analysis |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Plan Allowances
|
||||||
|
|
||||||
|
| Plan | Content Pieces/Month | Sites | Users |
|
||||||
|
|------|---------------------|-------|-------|
|
||||||
|
| Starter | 50 | 2 | 2 |
|
||||||
|
| Growth | 200 | 5 | 3 |
|
||||||
|
| Scale | 500 | Unlimited | 5 |
|
||||||
|
|
||||||
|
**Included with every content piece:**
|
||||||
|
- AI keyword clustering
|
||||||
|
- AI idea generation
|
||||||
|
- AI content writing (1000-2000 words)
|
||||||
|
- 3 images (1 featured + 2 in-article)
|
||||||
|
- Internal linking
|
||||||
|
- SEO optimization
|
||||||
|
- WordPress publishing
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Backend Soft Limits (Hidden from Users)
|
||||||
|
|
||||||
|
To prevent abuse, the backend enforces hidden limits:
|
||||||
|
|
||||||
|
| Limit | Starter | Growth | Scale |
|
||||||
|
|-------|---------|--------|-------|
|
||||||
|
| Keyword imports/mo | 500 | 2,000 | 5,000 |
|
||||||
|
| Clustering operations | 100 | 400 | 1,000 |
|
||||||
|
| Idea generations | 150 | 600 | 1,500 |
|
||||||
|
| Images generated | 200 | 800 | 2,000 |
|
||||||
|
|
||||||
|
If users hit these limits, they see: "You've reached your preparation limit for this month."
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Content Deduction Flow
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────────┐
|
||||||
|
│ CONTENT CREATION FLOW │
|
||||||
|
├─────────────────────────────────────────────────────────────────┤
|
||||||
|
│ │
|
||||||
|
│ User clicks Check Generate │
|
||||||
|
│ "Generate" ──────► Allowance ──────► Content │
|
||||||
|
│ │ │ │
|
||||||
|
│ │ Limit │ │
|
||||||
|
│ │ Reached ▼ │
|
||||||
|
│ ▼ Deduct 1 │
|
||||||
|
│ Show Upgrade Content │
|
||||||
|
│ Modal Piece │
|
||||||
|
│ │
|
||||||
|
└─────────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Operations & Credit Costs
|
||||||
|
|
||||||
|
### Planner Operations
|
||||||
|
|
||||||
|
| Operation | Credits | Type |
|
||||||
|
|-----------|---------|------|
|
||||||
|
| Add keyword | 0 | Free |
|
||||||
|
| Auto-cluster keywords | 1 | Idea |
|
||||||
|
| Generate content ideas | 1 per idea | Idea |
|
||||||
|
|
||||||
|
### Writer Operations
|
||||||
|
|
||||||
|
| Operation | Credits | Type |
|
||||||
|
|-----------|---------|------|
|
||||||
|
| Create task | 0 | Free |
|
||||||
|
| Generate content | 1 | Content |
|
||||||
|
| Regenerate content | 1 | Content |
|
||||||
|
| Generate images | 1 per image | Image |
|
||||||
|
| Regenerate image | 1 | Image |
|
||||||
|
| Edit content | 0 | Free |
|
||||||
|
|
||||||
|
### Automation Operations
|
||||||
|
|
||||||
|
| Operation | Credits | Type |
|
||||||
|
|-----------|---------|------|
|
||||||
|
| Run automation | Sum of operations | Mixed |
|
||||||
|
| Pause/resume | 0 | Free |
|
||||||
|
|
||||||
|
### Publisher Operations
|
||||||
|
|
||||||
|
| Operation | Credits | Type |
|
||||||
|
|-----------|---------|------|
|
||||||
|
| Publish to WordPress | 0 | Free |
|
||||||
|
| Sync from WordPress | 0 | Free |
|
||||||
|
|
||||||
|
### Optimizer Operations (Future)
|
||||||
|
|
||||||
|
| Operation | Credits | Type |
|
||||||
|
|-----------|---------|------|
|
||||||
|
| Optimize content | 1 | Optimization |
|
||||||
|
| Batch optimize | 1 per item | Optimization |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Database Models
|
||||||
|
|
||||||
|
### CreditBalance
|
||||||
|
|
||||||
|
```python
|
||||||
|
class CreditBalance(models.Model):
|
||||||
|
account = models.ForeignKey(Account)
|
||||||
|
site = models.ForeignKey(Site, null=True)
|
||||||
|
|
||||||
|
idea_credits = models.IntegerField(default=0)
|
||||||
|
content_credits = models.IntegerField(default=0)
|
||||||
|
image_credits = models.IntegerField(default=0)
|
||||||
|
optimization_credits = models.IntegerField(default=0)
|
||||||
|
|
||||||
|
period_start = models.DateField()
|
||||||
|
period_end = models.DateField()
|
||||||
|
```
|
||||||
|
|
||||||
|
### CreditUsage
|
||||||
|
|
||||||
|
```python
|
||||||
|
class CreditUsage(models.Model):
|
||||||
|
account = models.ForeignKey(Account)
|
||||||
|
site = models.ForeignKey(Site, null=True)
|
||||||
|
user = models.ForeignKey(User)
|
||||||
|
|
||||||
|
credit_type = models.CharField() # idea/content/image/optimization
|
||||||
|
amount = models.IntegerField()
|
||||||
|
operation = models.CharField() # generate_content, etc.
|
||||||
|
|
||||||
|
created_at = models.DateTimeField(auto_now_add=True)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Business Logic
|
||||||
|
|
||||||
|
### CreditService
|
||||||
|
|
||||||
|
Location: `backend/igny8_core/business/billing/services.py`
|
||||||
|
|
||||||
|
**Key Methods:**
|
||||||
|
|
||||||
|
```python
|
||||||
|
class CreditService:
|
||||||
|
def check_balance(account, site, credit_type, amount) -> bool:
|
||||||
|
"""Check if sufficient credits available"""
|
||||||
|
|
||||||
|
def deduct_credits(account, site, user, credit_type, amount, operation) -> bool:
|
||||||
|
"""Deduct credits and log usage"""
|
||||||
|
|
||||||
|
def get_balance(account, site) -> CreditBalance:
|
||||||
|
"""Get current balance"""
|
||||||
|
|
||||||
|
def reset_monthly_credits(account) -> None:
|
||||||
|
"""Reset credits at period start"""
|
||||||
|
|
||||||
|
def add_credits(account, credit_type, amount, reason) -> None:
|
||||||
|
"""Add credits (admin/purchase)"""
|
||||||
|
```
|
||||||
|
|
||||||
|
### Usage in AI Operations
|
||||||
|
|
||||||
|
```python
|
||||||
|
# In content generation service
|
||||||
|
def generate_content(task, user):
|
||||||
|
# 1. Check balance
|
||||||
|
if not credit_service.check_balance(
|
||||||
|
account=task.site.account,
|
||||||
|
site=task.site,
|
||||||
|
credit_type='content',
|
||||||
|
amount=1
|
||||||
|
):
|
||||||
|
raise InsufficientCreditsError()
|
||||||
|
|
||||||
|
# 2. Execute AI function
|
||||||
|
content = ai_engine.generate_content(task)
|
||||||
|
|
||||||
|
# 3. Deduct credits
|
||||||
|
credit_service.deduct_credits(
|
||||||
|
account=task.site.account,
|
||||||
|
site=task.site,
|
||||||
|
user=user,
|
||||||
|
credit_type='content',
|
||||||
|
amount=1,
|
||||||
|
operation='generate_content'
|
||||||
|
)
|
||||||
|
|
||||||
|
return content
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## API Responses
|
||||||
|
|
||||||
|
### Successful Deduction
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"success": true,
|
||||||
|
"data": { ... },
|
||||||
|
"credits_used": {
|
||||||
|
"type": "content",
|
||||||
|
"amount": 1
|
||||||
|
},
|
||||||
|
"balance": {
|
||||||
|
"content_credits": 49
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Insufficient Credits
|
||||||
|
|
||||||
|
```json
|
||||||
|
HTTP 402 Payment Required
|
||||||
|
|
||||||
|
{
|
||||||
|
"success": false,
|
||||||
|
"error": "Insufficient content credits",
|
||||||
|
"code": "INSUFFICIENT_CREDITS",
|
||||||
|
"required": 1,
|
||||||
|
"available": 0
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Frontend Handling
|
||||||
|
|
||||||
|
### Balance Display
|
||||||
|
|
||||||
|
- Header shows credit balances
|
||||||
|
- Updates after each operation
|
||||||
|
- Warning at low balance (< 10%)
|
||||||
|
|
||||||
|
### Error Handling
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// In writer store
|
||||||
|
async generateContent(taskId: string) {
|
||||||
|
try {
|
||||||
|
const response = await api.generateContent(taskId);
|
||||||
|
// Update billing store
|
||||||
|
billingStore.fetchBalance();
|
||||||
|
return response;
|
||||||
|
} catch (error) {
|
||||||
|
if (error.code === 'INSUFFICIENT_CREDITS') {
|
||||||
|
// Show upgrade modal
|
||||||
|
uiStore.showUpgradeModal();
|
||||||
|
}
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Usage Tracking
|
||||||
|
|
||||||
|
### Usage Summary Endpoint
|
||||||
|
|
||||||
|
```
|
||||||
|
GET /api/v1/billing/usage/summary/?period=month
|
||||||
|
```
|
||||||
|
|
||||||
|
Response:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"period": "2025-01",
|
||||||
|
"usage": {
|
||||||
|
"idea_credits": 45,
|
||||||
|
"content_credits": 23,
|
||||||
|
"image_credits": 67,
|
||||||
|
"optimization_credits": 0
|
||||||
|
},
|
||||||
|
"by_operation": {
|
||||||
|
"auto_cluster": 12,
|
||||||
|
"generate_ideas": 33,
|
||||||
|
"generate_content": 23,
|
||||||
|
"generate_images": 67
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Automation Credit Estimation
|
||||||
|
|
||||||
|
Before running automation:
|
||||||
|
|
||||||
|
```
|
||||||
|
GET /api/v1/automation/estimate/?site_id=...
|
||||||
|
```
|
||||||
|
|
||||||
|
Response:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"estimated_credits": {
|
||||||
|
"idea_credits": 25,
|
||||||
|
"content_credits": 10,
|
||||||
|
"image_credits": 30
|
||||||
|
},
|
||||||
|
"stages": {
|
||||||
|
"clustering": 5,
|
||||||
|
"ideas": 20,
|
||||||
|
"content": 10,
|
||||||
|
"images": 30
|
||||||
|
},
|
||||||
|
"has_sufficient_credits": true
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Credit Reset
|
||||||
|
|
||||||
|
Credits reset monthly based on billing cycle:
|
||||||
|
|
||||||
|
1. **Monthly Reset Job** runs at period end
|
||||||
|
2. **Unused credits** do not roll over
|
||||||
|
3. **Purchased credits** may have different expiry
|
||||||
|
|
||||||
|
### Celery Task
|
||||||
|
|
||||||
|
```python
|
||||||
|
@celery.task
|
||||||
|
def reset_monthly_credits():
|
||||||
|
"""
|
||||||
|
Run daily, resets credits for accounts
|
||||||
|
whose period_end is today
|
||||||
|
"""
|
||||||
|
today = date.today()
|
||||||
|
balances = CreditBalance.objects.filter(period_end=today)
|
||||||
|
|
||||||
|
for balance in balances:
|
||||||
|
credit_service.reset_monthly_credits(balance.account)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Admin Operations
|
||||||
|
|
||||||
|
### Manual Credit Adjustment
|
||||||
|
|
||||||
|
Via Django Admin or API:
|
||||||
|
|
||||||
|
```python
|
||||||
|
# Add credits
|
||||||
|
credit_service.add_credits(
|
||||||
|
account=account,
|
||||||
|
credit_type='content',
|
||||||
|
amount=100,
|
||||||
|
reason='Customer support adjustment'
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Usage Audit
|
||||||
|
|
||||||
|
All credit changes logged in `CreditUsage` with:
|
||||||
|
- Timestamp
|
||||||
|
- User who triggered
|
||||||
|
- Operation type
|
||||||
|
- Amount deducted
|
||||||
|
- Related object ID
|
||||||
@@ -1,69 +0,0 @@
|
|||||||
# Billing Lifecycle
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Detail how credits, plans, subscriptions, invoices, and payments flow through the system, and where deductions/additions are enforced.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Models: `backend/igny8_core/business/billing/models.py` (`CreditTransaction`, `CreditUsageLog`, `CreditCostConfig`, `Invoice`, `Payment`, `CreditPackage`, `PaymentMethodConfig`, `AccountPaymentMethod`)
|
|
||||||
- Services: `backend/igny8_core/business/billing/services/credit_service.py`, `services/invoice_service.py`, `services/payment_service.py`
|
|
||||||
- Views/Endpoints: `backend/igny8_core/modules/billing/views.py`
|
|
||||||
- Frontend billing pages: `frontend/src/pages/account/{AccountBillingPage.tsx,PlansAndBillingPage.tsx,PurchaseCreditsPage.tsx}`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Track credit balance, record usage, and enforce costs for AI/automation actions.
|
|
||||||
- Manage plans/subscriptions, generate invoices, and record payments.
|
|
||||||
- Expose account-scoped billing data (balance, usage, invoices, payments, packages, payment methods).
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Credit balance: stored on `Account.credits`; `CreditService` adds/deducts via `CreditTransaction` and logs usage in `CreditUsageLog`.
|
|
||||||
- Costs: read from `CreditCostConfig` or hardcoded `CREDIT_COSTS` fallbacks inside `credit_service.py`.
|
|
||||||
- Deductions: called from AI/automation flows to ensure sufficient credits; `InsufficientCreditsError` thrown on shortage.
|
|
||||||
- Plans/Subscriptions: `Plan` defines price, billing_cycle, included_credits, max sites/sectors/users; `Subscription` links account to plan. Plan selection handled via billing endpoints and frontend plan tabs.
|
|
||||||
- Invoices: `InvoiceService` creates invoices for subscriptions and credit packages; generates unique invoice numbers and line items.
|
|
||||||
- Payments: `PaymentService` records payments; manual payments stored with status (pending/processing/etc.); payment methods configured via `PaymentMethodConfig`/`AccountPaymentMethod`.
|
|
||||||
- Packages: `CreditPackage` defines purchasable credit bundles; purchasing triggers invoice/payment flows.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Billing: `CreditTransaction`, `CreditUsageLog`, `CreditCostConfig`, `Invoice`, `Payment`, `CreditPackage`, `PaymentMethodConfig`, `AccountPaymentMethod`.
|
|
||||||
- Plan: `Plan`, `Subscription` (in `auth/models.py`).
|
|
||||||
- Account linkage: `Account` holds `credits` and `plan`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Balance/Usage read: billing endpoints return account-scoped balance and usage summaries.
|
|
||||||
- AI/Automation call: service invokes `CreditService.deduct_credits_for_action` → creates `CreditTransaction` + `CreditUsageLog`.
|
|
||||||
- Purchase credits: frontend calls billing endpoint → `CreditService.add_credits` + `InvoiceService` for package; payment recorded via `PaymentService` (manual/other).
|
|
||||||
- Subscription change: endpoint updates `Subscription` + `Account.plan`, generates invoice as needed; payment recorded if required.
|
|
||||||
- Invoice download: `modules/billing/views.py` exposes invoice retrieval; frontend uses `downloadInvoicePDF`.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Automation/writer AI calls depend on credit checks; insufficient balance blocks operations.
|
|
||||||
- Account settings feed invoice billing details; payments/invoices are tied to account.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Payment status: `pending` → `processing/pending_approval` → `succeeded/completed` or `failed/refunded/cancelled/void/uncollectible`.
|
|
||||||
- Subscription status: `active`/`cancelled` (persisted in `Subscription`); plan reference on account updates accordingly.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- `InsufficientCreditsError` for low balance; surfaces to API as error response.
|
|
||||||
- Payment/Invoice service raise validation errors (e.g., already paid invoice).
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All billing queries filter by `request.user.account`; viewsets inherit from `AccountModelViewSet`.
|
|
||||||
- Payment methods, invoices, payments, credit transactions are account-scoped; no cross-tenant access.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Costs derived from config constants or `CreditCostConfig`.
|
|
||||||
- Included credits and limits come from `Plan`; extra purchases via `CreditPackage`.
|
|
||||||
- Credits must be available before AI/automation runs proceed.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- None dedicated; billing operations are request-driven. Automation tasks carry account context to deduct credits within Celery runs.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- CreditService centralizes deductions to avoid scattered logic.
|
|
||||||
- Manual payment flow avoids storing sensitive data; relies on transaction_reference and admin approval.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- When adding new billable actions, route them through `CreditService` with a defined cost key.
|
|
||||||
- To change pricing, update `CreditCostConfig` data or the `CREDIT_COSTS` fallback map.
|
|
||||||
- Keep all billing endpoints inheriting `AccountModelViewSet` to maintain isolation.
|
|
||||||
@@ -1,77 +0,0 @@
|
|||||||
# User Flow Overview
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe the end-to-end journey from signup through planning, writing, automation, publishing, and billing within IGNY8, mapping each step to concrete backend/frontend modules so engineers can navigate without scanning code.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Auth & account: `backend/igny8_core/auth/{views.py,serializers.py,models.py,middleware.py}`, `frontend/src/store/authStore.ts`, routes in `frontend/src/App.tsx`
|
|
||||||
- Planner: `backend/igny8_core/modules/planner/views.py`, `business/planning/models.py`, `frontend/src/pages/Planner/*`
|
|
||||||
- Writer: `backend/igny8_core/modules/writer/views.py`, `business/content/models.py`, `frontend/src/pages/Writer/*`
|
|
||||||
- Automation: `backend/igny8_core/business/automation/{services/automation_service.py,tasks.py,views.py}`, `frontend/src/pages/Automation/AutomationPage.tsx`
|
|
||||||
- Publishing/Integration: `backend/igny8_core/business/integration/{models.py,services/integration_service.py}`, `modules/integration/views.py`, WordPress plugin endpoints consumed via `/api/v1/integration/`; `frontend/src/pages/Sites/*`
|
|
||||||
- Billing: `backend/igny8_core/business/billing/{models.py,services/credit_service.py,services/invoice_service.py,views.py}`, `frontend/src/pages/account/*`
|
|
||||||
- Tenancy enforcement: `backend/igny8_core/auth/middleware.py`, `backend/igny8_core/api/base.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Authenticate users and attach account context to every request.
|
|
||||||
- Let users plan keywords/clusters/ideas, create tasks and content, optionally automate all seven pipeline stages.
|
|
||||||
- Manage sites/sectors and connect WordPress for publishing/sync.
|
|
||||||
- Track credits, plans, subscriptions, invoices, and payments through billing services.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Signup/Login: `auth/views.py` issues JWTs; `AccountContextMiddleware` sets `request.account`; frontend `authStore` stores tokens and refreshes them.
|
|
||||||
- Planning: `KeywordViewSet`, `ClusterViewSet`, `ContentIdeasViewSet` create/list/update scoped by `SiteSectorModelViewSet` filters; frontend planner pages drive these endpoints.
|
|
||||||
- Writing: `TasksViewSet`, `ContentViewSet`, `ImagesViewSet` manage tasks/content/images; AI generation endpoints trigger Celery-backed functions.
|
|
||||||
- Automation: `AutomationViewSet` + `automation_service.py` orchestrate 7 stages (keywords→clusters→ideas→tasks→content→image-prompts→images/manual review) with pause/resume and credit estimation; Celery tasks `run_automation_task`/`resume_automation_task` execute runs.
|
|
||||||
- Publishing/Integration: `IntegrationViewSet` handles WP connection tests and sync; WP plugin sends/receives data via API key; content publish endpoints in writer module update WordPress via integration services.
|
|
||||||
- Billing: credit balances and costs computed in `credit_service.py`; invoices via `invoice_service.py`; payments via `payment_service.py`; endpoints in `modules/billing/views.py` feed frontend billing pages.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Tenancy: `Account`, `Site`, `Sector` (plus `SiteUserAccess`), base models `AccountBaseModel`/`SiteSectorBaseModel`.
|
|
||||||
- Planning: `Keywords`, `Clusters`, `ContentIdeas`.
|
|
||||||
- Writing: `Tasks`, `Content`, `Images`.
|
|
||||||
- Automation: `AutomationConfig`, `AutomationRun`.
|
|
||||||
- Billing: `CreditTransaction`, `CreditUsageLog`, `CreditCostConfig`, `Invoice`, `Payment`, `CreditPackage`, `Subscription`, `Plan`.
|
|
||||||
- Integration: `SiteIntegration`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
1) User signs in → JWT stored → `AccountContextMiddleware` populates `request.account`.
|
|
||||||
2) Tenant creates sites/sectors (`SiteViewSet`), selects industry/sectors, optionally connects WordPress.
|
|
||||||
3) Planner: upload/enter keywords → cluster → generate ideas (manual or via automation/AI functions).
|
|
||||||
4) Writer: create tasks from ideas, generate content/images (manual endpoints or automation stages).
|
|
||||||
5) Publish: send to WordPress via integration endpoints or automation publish step; WP plugin syncs back statuses.
|
|
||||||
6) Automation (optional): run 7-stage pipeline via `AutomationViewSet` + Celery tasks; pause/resume supported.
|
|
||||||
7) Billing: credits deducted per AI/pipeline usage (`credit_service`), invoices/payments recorded; users view in billing pages.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Automation invokes planner/writer endpoints/services and logs credit estimates.
|
|
||||||
- Billing hooks into automation and writer AI calls via credit deduction utilities.
|
|
||||||
- Integration uses site/account context and WP API key for sync; writer publish flows depend on integration configuration.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Account status (`active/suspended/trial/cancelled`) governs access; plan/subscription affect limits.
|
|
||||||
- Tasks/content/images status transitions handled in writer endpoints; automation run status moves through `created/running/paused/completed/failed`.
|
|
||||||
- Site activation via `set_active`; sectors toggled via `select_sectors`.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Unified API responses via `api/response.py`; DRF exception handler configured in settings; frontend shows toasts/banners.
|
|
||||||
- Automation errors logged via `automation_logger`; tasks wrapped in Celery with retries where defined.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- `AccountContextMiddleware` sets `request.account`; base viewsets filter by account/site/sector; API key auth sets account from `Site.wp_api_key`; public site slug reads limited to active sites.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- AI/automation uses `CreditService.deduct_credits_for_action`; credit balance required before runs; plans/subscriptions define included credits and sector/site limits.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Celery tasks: automation runs (`run_automation_task`, `resume_automation_task`), scheduler `check_scheduled_automations`, AI functions (`ai/tasks.py`), publishing tasks (`tasks/wordpress_publishing.py`).
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Strong tenant isolation via middleware + filtered querysets.
|
|
||||||
- Automation relies on polling and Celery; frontend automation page polls every 5s during runs.
|
|
||||||
- Billing is authoritative in backend; frontend is read-only except initiating purchases/subscriptions.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Trace user-visible flows by following router → page component → service call → backend viewset.
|
|
||||||
- When adding steps, ensure credit deductions and tenancy filters are applied in the corresponding backend service/viewset.
|
|
||||||
- Keep WP integration changes consistent with API key auth and `SiteIntegration` schema.
|
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -1,38 +0,0 @@
|
|||||||
# Archived Documentation
|
|
||||||
|
|
||||||
**Purpose:** Historical reference from previous documentation systems
|
|
||||||
|
|
||||||
## Contents
|
|
||||||
|
|
||||||
### master-docs-original/
|
|
||||||
Original master documentation structure (pre-December 2024 consolidation)
|
|
||||||
- Comprehensive but scattered across many files
|
|
||||||
- Retained for historical reference
|
|
||||||
- **Do not use for active development**
|
|
||||||
|
|
||||||
### old-docs-original/
|
|
||||||
Legacy documentation from earlier iterations
|
|
||||||
- Contains older API references
|
|
||||||
- Has some WordPress-specific guides
|
|
||||||
- **Do not use for active development**
|
|
||||||
|
|
||||||
## Current Documentation
|
|
||||||
|
|
||||||
All active documentation has been consolidated into:
|
|
||||||
- `/docs/` - Single source of truth
|
|
||||||
- See `/docs/README.md` for navigation
|
|
||||||
|
|
||||||
## When to Reference Archived Docs
|
|
||||||
|
|
||||||
✅ Historical context for old decisions
|
|
||||||
✅ Finding deprecated features
|
|
||||||
✅ Understanding system evolution
|
|
||||||
|
|
||||||
❌ NOT for current development
|
|
||||||
❌ NOT for API references
|
|
||||||
❌ NOT for implementation guides
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Last Archived:** December 9, 2024
|
|
||||||
**Reason:** Documentation consolidation project
|
|
||||||
@@ -1,79 +0,0 @@
|
|||||||
# System Architecture Overview
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe how IGNY8 is structured across backend, frontend, and integrations, grounded in the current codebase. Covers core services, middleware, and platform composition.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Backend project root: `backend/igny8_core/`
|
|
||||||
- Settings and service wiring: `backend/igny8_core/settings.py`
|
|
||||||
- URL routing: `backend/igny8_core/urls.py`
|
|
||||||
- Middleware: `backend/igny8_core/middleware/request_id.py`, `backend/igny8_core/middleware/resource_tracker.py`, `backend/igny8_core/auth/middleware.py`
|
|
||||||
- Auth models and tenancy bases: `backend/igny8_core/auth/models.py`
|
|
||||||
- DRF base behaviors: `backend/igny8_core/api/base.py`
|
|
||||||
- Custom auth classes: `backend/igny8_core/api/authentication.py`
|
|
||||||
- Frontend SPA: `frontend/` (Vite + React; dependencies in `frontend/package.json`)
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Django/DRF backend providing multi-tenant APIs for planner, writer, system, billing, automation, linker, optimizer, publisher, and integration modules.
|
|
||||||
- Middleware adds per-request IDs, tenant context, and optional resource tracking for admin diagnostics.
|
|
||||||
- REST API routing under `/api/v1/*` with unified response/error handling and scoped throttling.
|
|
||||||
- Celery-backed async work (configured in settings) for AI, automation, and publishing.
|
|
||||||
- React/Vite frontend consuming the API; authentication via JWT or session; API key support for WordPress bridge.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Backend apps registered in `INSTALLED_APPS` include auth, AI framework, planner, writer, system, billing, automation, optimization, publishing, integration, linker, optimizer, and publisher modules; these are loaded via Django app configs in `settings.py`.
|
|
||||||
- Middleware order enforces security, CORS, session, CSRF, Django auth, then custom layers: request ID, account context, resource tracking, messages, and clickjacking protection.
|
|
||||||
- URL map (`urls.py`) exposes admin, CSV admin utilities for industries/seed keywords, and module routers for auth, account, planner, writer, system, billing (user + admin), automation, linker, optimizer, publisher, and integration. OpenAPI docs available at `/api/docs` and `/api/redoc`.
|
|
||||||
- REST framework defaults use custom pagination, filtering, search, ordering, and a custom exception handler (enabled unless `IGNY8_USE_UNIFIED_EXCEPTION_HANDLER` is false). Authentication stack orders API key, JWT, CSRF-exempt session, then basic auth. Throttle scopes are predefined per domain (AI, content, auth, planner, writer, system, billing, linker, optimizer, integration).
|
|
||||||
- CORS allows the IGNY8 domains plus local development hosts; credentials and specific debug headers are permitted, and request/resource tracking IDs are exposed in response headers.
|
|
||||||
- Celery is configured to use Redis by default for broker and result backend, with JSON serialization, task time limits, and single-prefetch workers.
|
|
||||||
- Logging writes to console and rotating files for publish/sync, WordPress API calls, and webhooks, with request IDs available via middleware.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code, just explanation)
|
|
||||||
- Multi-tenancy base classes (`AccountBaseModel`, `SiteSectorBaseModel`) add `account`, `site`, and `sector` scoping plus validation; defined in `auth/models.py`.
|
|
||||||
- Core tenancy entities: `Account`, `Plan`, `Subscription`, `Site`, `Sector`, `Industry`, `IndustrySector`, `SeedKeyword`, `SiteUserAccess`, `User`, and `PasswordResetToken` in `auth/models.py`.
|
|
||||||
- These bases are consumed by downstream modules (planner, writer, billing, automation, etc.) to enforce tenant, site, and sector ownership.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Incoming HTTP requests enter Django middleware: security → WhiteNoise → CORS → session → common/CSRF → Django auth → `RequestIDMiddleware` (assigns `X-Request-ID`) → `AccountContextMiddleware` (sets `request.account` via session or JWT) → `ResourceTrackingMiddleware` (when enabled for admins) → messages → clickjacking.
|
|
||||||
- DRF viewsets inherit from base classes in `api/base.py` to auto-filter querysets by account (and site/sector where applicable) and emit unified responses.
|
|
||||||
- URL dispatch routes to module-specific routers under `/api/v1/*`. CSV admin helpers are mounted before admin to avoid routing conflicts.
|
|
||||||
- Background tasks are dispatched via Celery using Redis endpoints from `settings.py`.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Auth middleware sets `request.account` consumed by `AccountModelViewSet` and `SiteSectorModelViewSet` to filter data.
|
|
||||||
- API key auth (WordPress bridge) sets both `request.account` and `request.site` for integration endpoints.
|
|
||||||
- Resource tracking middleware exposes metrics via cache keyed by the request-specific tracking ID added to responses.
|
|
||||||
- OpenAPI generation (drf-spectacular) documents all modules and respects unified response schemas.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Request lifecycle: ID assignment → tenant resolution → optional resource profiling → viewset execution → unified response with optional pagination and request/resource IDs in headers.
|
|
||||||
- Tenancy lifecycle: Account/Plan/Subscription fields in models track status and retention; soft delete support on models that implement it.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Global DRF exception handler (custom when enabled) wraps errors into unified JSON with `success=false`.
|
|
||||||
- Account middleware denies access when account or plan is missing/inactive, returning structured JSON and logging out session users.
|
|
||||||
- Viewset overrides in `api/base.py` return unified error payloads for validation and 404/500 cases.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Account is injected via middleware (session or JWT). Base viewsets filter querysets by `account`, unless the user is admin/developer/system account (override path).
|
|
||||||
- `SiteSectorModelViewSet` adds site/sector filtering when models carry those fields; validation ensures site/sector belong to the same account.
|
|
||||||
- Role helpers (`User.is_admin_or_developer`, `User.is_system_account_user`) allow bypass for privileged users; otherwise, data is restricted to the resolved account (and site/sector for derived models).
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Plan and account billing fields live in `auth/models.py` (`Plan.included_credits`, `Account.credits`, Stripe IDs). Status enforcement occurs in `AccountContextMiddleware` by requiring an active plan; billing modules implement credit logic (covered in backend/billing docs).
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Celery configuration in `settings.py` sets Redis broker/backend, JSON serialization, time limits, and worker tuning. Task modules (AI, automation, publishing) use this setup; scheduling uses Celery Beat state.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Middleware-first tenant resolution ensures consistent scoping before view logic.
|
|
||||||
- Admin/developer/system-account overrides allow cross-tenant operations for ops while protecting system accounts from deletion.
|
|
||||||
- Unified API responses and throttling scopes enforce consistent client behavior and rate safety.
|
|
||||||
- Redis-backed Celery keeps async workloads out of request path with strict time limits.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Add new apps to `INSTALLED_APPS` and mount URLs under `/api/v1/*` in `urls.py`.
|
|
||||||
- Inherit from `AccountModelViewSet` or `SiteSectorModelViewSet` to automatically enforce tenant/site/sector scoping and unified responses.
|
|
||||||
- Use existing middleware; do not reorder request ID or account context layers, as downstream views rely on them.
|
|
||||||
- Configure environment via `settings.py` variables (DB, JWT, Celery, CORS, Stripe/PayPal) rather than hardcoding values.
|
|
||||||
@@ -1,61 +0,0 @@
|
|||||||
# Tech Stack
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Document the concrete technologies and dependencies in use across backend and frontend as defined in the repository.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Backend dependency manifest: `backend/requirements.txt`
|
|
||||||
- Backend settings (framework integration): `backend/igny8_core/settings.py`
|
|
||||||
- Frontend dependency manifest: `frontend/package.json`
|
|
||||||
- Frontend build tooling: `frontend/vite.config.ts`, `frontend/tsconfig*.json`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Backend: Django 5.x with DRF for APIs, Celery for async tasks, Redis for broker/result, PostgreSQL or SQLite for data, drf-spectacular for OpenAPI, Stripe/PayPal configs for billing, WhiteNoise for static serving.
|
|
||||||
- Frontend: React 19 with Vite, TypeScript, TailwindCSS, Zustand state, React Router 7, ApexCharts and FullCalendar for UI widgets.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Backend settings wire DRF pagination, filters, authentication (API key, JWT, session, basic), throttling, schema generation, CORS, Celery, logging, and Stripe/PayPal credentials. Static assets are served via WhiteNoise; admin uses Django contrib.
|
|
||||||
- `requirements.txt` enumerates runtime libs: Django, gunicorn, psycopg2-binary (PostgreSQL), redis, WhiteNoise, DRF, django-filter, django-cors-headers, PyJWT, requests, Celery, BeautifulSoup4, psutil, docker (for ops scripts), drf-spectacular, and stripe.
|
|
||||||
- Frontend `package.json` pins React 19, React Router 7, Zustand 5, Vite 6, TailwindCSS 4, ApexCharts 4, FullCalendar 6, react-dnd, dropzone, lucide/react-heroicons, and testing/tooling (Vitest, Testing Library, ESLint).
|
|
||||||
- Build scripts use Vite for dev and production builds, with separate marketing mode; tests via Vitest; lint via ESLint; type-check via `tsc -b`.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Not applicable; this file tracks technology components rather than domain models.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Backend runs under Django/DRF with middleware and installed apps per `settings.py`. ASGI/WSGI entrypoints in `igny8_core/asgi.py` and `igny8_core/wsgi.py` (default WSGI via gunicorn).
|
|
||||||
- Celery worker/beat use Redis URLs from `settings.py` and respect JSON serialization/time limits.
|
|
||||||
- Frontend builds with Vite, consuming environment variables defined via Vite conventions.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- DRF auth classes depend on PyJWT and custom utilities for token handling.
|
|
||||||
- Celery tasks in AI/automation/publishing rely on Redis connectivity and the configured serializer/time limits.
|
|
||||||
- Stripe/PayPal keys in settings are consumed by billing modules.
|
|
||||||
- Frontend API calls rely on the DRF endpoints exposed in `igny8_core/urls.py`.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Dependency-driven: none beyond the build/runtime phases (install → build → serve).
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Backend error handling configured via custom DRF exception handler (enabled by default) and logging setup in `settings.py`.
|
|
||||||
- Frontend build/test errors are surfaced through Vite/TypeScript/Vitest/ESLint tooling.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Implemented at runtime by backend middleware and viewsets; the tech stack provides JWT, session, and API key support to carry tenant context.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Stripe and PayPal keys in `settings.py` enable billing integrations; credit logic is implemented in billing modules (documented elsewhere).
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Celery configured in `settings.py` with Redis broker/backend, JSON serialization, and task limits; beat scheduling persists in `celerybeat-schedule`.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Keep dependency manifests authoritative (`requirements.txt`, `package.json`).
|
|
||||||
- Redis is the default async backbone; Postgres is the default DB with SQLite fallback for local/dev.
|
|
||||||
- Vite + React 19 selected for fast dev/build; TailwindCSS 4 used for styling; Zustand for state.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Add backend dependencies to `requirements.txt` and pin versions appropriately; update settings if new middleware/auth is added.
|
|
||||||
- Add frontend dependencies to `package.json`; run `npm install` and ensure Vite/TSC builds remain clean.
|
|
||||||
- Respect configured auth stack (API key → JWT → session) when adding API clients.
|
|
||||||
- Keep CORS and env vars aligned with the domains/ports in use for local and production.
|
|
||||||
@@ -1,79 +0,0 @@
|
|||||||
# Multitenancy Model
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Explain how tenant, site, and sector isolation is enforced across models, middleware, and viewsets, based on the current implementation.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Tenant base models: `backend/igny8_core/auth/models.py` (`AccountBaseModel`, `SiteSectorBaseModel`)
|
|
||||||
- Core entities: `backend/igny8_core/auth/models.py` (`Account`, `Plan`, `Site`, `Sector`, `Industry`, `IndustrySector`, `SeedKeyword`, `SiteUserAccess`, `User`)
|
|
||||||
- Middleware for context: `backend/igny8_core/auth/middleware.py`
|
|
||||||
- DRF base viewsets: `backend/igny8_core/api/base.py`
|
|
||||||
- Auth utilities and JWT: `backend/igny8_core/api/authentication.py`, `backend/igny8_core/auth/utils.py`
|
|
||||||
- URL routing (module mounting): `backend/igny8_core/urls.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Enforce per-account isolation for all models carrying an `account` FK.
|
|
||||||
- Enforce per-site and per-sector scoping for content models via `SiteSectorBaseModel`.
|
|
||||||
- Inject tenant context on every request (session or JWT/API key), then apply scoping in base viewsets.
|
|
||||||
- Allow controlled overrides for admins, developers, and system accounts.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- `AccountBaseModel` adds an `account` FK plus timestamps and indexes; all tenant-scoped models inherit this to guarantee account linkage.
|
|
||||||
- `SiteSectorBaseModel` extends `AccountBaseModel` with `site` and `sector` FKs, indexes on `(account, site, sector)`, and a save hook that sets `account` from `site` and validates that `sector` belongs to the same `site`; raises validation errors on mismatch.
|
|
||||||
- `AccountContextMiddleware` sets `request.account` by refreshing the authenticated session user (with account and plan) or by decoding JWT tokens; it rejects requests when account is missing or plan is inactive, returning structured JSON and logging out session users. It skips admin and auth endpoints to avoid interference.
|
|
||||||
- JWT authentication (`api/authentication.py`) decodes tokens and sets `request.account` from the token payload; API key authentication sets both `request.account` and `request.site` for WordPress bridge calls.
|
|
||||||
- `AccountModelViewSet` auto-filters querysets by `account` when models expose that field. It bypasses filtering for admins/developers/system-account users; otherwise, it uses `request.account` or falls back to the authenticated user’s account. Creates set `account` on save when present.
|
|
||||||
- `SiteSectorModelViewSet` extends the above to filter by site/sector if those fields exist, using request query parameters and tenancy context.
|
|
||||||
- `User` role helpers (`is_admin_or_developer`, `is_system_account_user`) and account checks gate override behavior.
|
|
||||||
- `SiteUserAccess` provides explicit per-site access for non-admin roles; `User.get_accessible_sites` respects system account, developer, owner/admin, and granted access rules.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `Account`: tenant container with plan, credits, billing fields, status, and retention settings.
|
|
||||||
- `Plan`: defines limits (max users/sites/industries/author profiles), credit inclusion, Stripe IDs.
|
|
||||||
- `Site`: belongs to an account, optionally an industry; includes status, hosting type, legacy WP fields, and SEO metadata.
|
|
||||||
- `Sector`: belongs to a site and industry sector template; enforces account alignment and plan-based max-sector validation.
|
|
||||||
- `Industry`, `IndustrySector`, `SeedKeyword`: global reference data not bound to a single account.
|
|
||||||
- `SiteUserAccess`: explicit many-to-many grants between users and sites.
|
|
||||||
- `User`: links to account with role-based access flags.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Request enters middleware; `AccountContextMiddleware` determines `request.account` (session or JWT/API key), validating plan status.
|
|
||||||
- Viewsets inheriting from `AccountModelViewSet`/`SiteSectorModelViewSet` filter querysets by `account` (and site/sector when present) before pagination/serialization.
|
|
||||||
- Object creation sets `account` automatically when the serializer’s model has that field; site/sector-based models validate alignment on save.
|
|
||||||
- Admin/developer/system-account users skip account filtering; other users remain constrained.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- All module viewsets depend on the base viewsets for scoping.
|
|
||||||
- Automation, planner, writer, billing, linker, optimizer, publisher, and integration models inherit from the tenancy bases to enforce account/site/sector ownership.
|
|
||||||
- API key flows for WordPress set `request.site`, enabling integration-specific logic to run in a site-aware context.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Account status changes (active, suspended, trial, cancelled) and plan activation directly affect access through middleware plan validation.
|
|
||||||
- Sector creation enforces plan-based limits for active sectors per site.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Middleware returns JSON errors for missing account or inactive plan, with HTTP 403 or 402 semantics and logs out session users.
|
|
||||||
- Base viewsets wrap CRUD operations in unified responses; validation failures or missing objects are returned in structured error payloads.
|
|
||||||
- Save-time validation on `SiteSectorBaseModel` and `Sector` raises validation errors when site/sector alignment or plan limits are violated.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Every tenant-scoped model carries `account`; site/sector-aware models carry `site` and `sector` and must align to the same account.
|
|
||||||
- Middleware populates `request.account`; base viewsets enforce filtering unless the user is an admin/developer/system-account member.
|
|
||||||
- System accounts (`aws-admin`, `default-account`, `default`) and privileged roles can bypass scoping; protected from deletion via guard clauses.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Middleware requires an active plan before allowing requests (except auth/admin paths). Credits, charges, and plan enforcement are handled in billing modules (documented elsewhere).
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Celery tasks inherit tenant context via payloads supplied by calling viewsets/services; the tenancy bases ensure stored records retain `account`/`site`/`sector`.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Tenancy is enforced as early as middleware to avoid leakage in view logic.
|
|
||||||
- Base viewsets centralize scoping and unified responses to reduce duplication across modules.
|
|
||||||
- Role-based overrides exist for ops and system accounts; safeguards prevent system account deletion.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Inherit from `AccountBaseModel` or `SiteSectorBaseModel` for any new tenant/site/sector data models.
|
|
||||||
- Inherit viewsets from `AccountModelViewSet` or `SiteSectorModelViewSet` to get automatic scoping and unified responses.
|
|
||||||
- Do not bypass `AccountContextMiddleware`; ensure new endpoints live under `/api/v1/*` and rely on the auth stack (API key → JWT → session).
|
|
||||||
- Validate that new background tasks carry account/site/sector identifiers so downstream saves remain scoped.
|
|
||||||
@@ -1,73 +0,0 @@
|
|||||||
# Identity and Authentication
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Document how user identity, JWT handling, API keys, and session flows work, including middleware and validation rules.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- JWT utilities: `backend/igny8_core/auth/utils.py`
|
|
||||||
- Account context middleware: `backend/igny8_core/auth/middleware.py`
|
|
||||||
- DRF authentication classes: `backend/igny8_core/api/authentication.py`
|
|
||||||
- DRF settings for auth/throttle: `backend/igny8_core/settings.py`
|
|
||||||
- User model and roles: `backend/igny8_core/auth/models.py`
|
|
||||||
- Auth URLs and views: `backend/igny8_core/auth/urls.py`, `backend/igny8_core/auth/views.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Support multiple auth mechanisms: API key (WordPress bridge), JWT bearer tokens, session auth without CSRF for APIs, and basic auth fallback.
|
|
||||||
- Populate tenant/site context alongside user identity so downstream viewsets enforce isolation.
|
|
||||||
- Enforce active account/plan presence before serving protected endpoints (except admin/auth routes).
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Authentication order (DRF `DEFAULT_AUTHENTICATION_CLASSES` in `settings.py`): API key → JWT → CSRF-exempt session → basic auth. The first class that authenticates sets `request.user`; `request.account` may also be set by API key or JWT.
|
|
||||||
- API Key flow (`APIKeyAuthentication`): expects `Authorization: Bearer <api_key>` that is not JWT-like; finds an active `Site` with `wp_api_key`, loads its `account`, and selects an active user (owner preferred, else any active developer/owner/admin). Sets `request.account` and `request.site`. Rejects short/invalid keys; returns an auth failure if no active user exists.
|
|
||||||
- JWT flow (`JWTAuthentication`): expects `Authorization: Bearer <jwt>`; decodes via `auth.utils.decode_token`; only accepts tokens with `type == access`. Retrieves `User` by `user_id`; optional `account_id` is resolved to `Account` and set on `request.account`. Invalid/expired tokens fall through to other auth classes.
|
|
||||||
- Session flow (`CSRFExemptSessionAuthentication`): uses Django session cookies without CSRF enforcement for API calls.
|
|
||||||
- Basic auth: last resort; does not set tenant context.
|
|
||||||
- Token utilities (`auth/utils.py`) generate and decode access/refresh tokens using expiries from settings (`JWT_ACCESS_TOKEN_EXPIRY`, `JWT_REFRESH_TOKEN_EXPIRY`), embedding `user_id`, `account_id`, `email`, issued/expiry timestamps, and token `type`.
|
|
||||||
- Middleware (`AccountContextMiddleware`) runs on every request except admin/auth paths: refreshes session users from DB to pick up current account/plan, validates presence of an active plan, sets `request.account`, and logs out session users when invalid. For JWT-bearing requests it decodes the token directly and sets `request.account`. If account/plan is missing or inactive, it returns JSON with `success=false` and appropriate HTTP status.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `User` with `role` and `account` FKs in `auth/models.py`.
|
|
||||||
- `Account` with plan and billing fields; plan status is used for access gating.
|
|
||||||
- `Site` with `wp_api_key` for API key auth; `SiteUserAccess` for per-site grants.
|
|
||||||
- `PasswordResetToken` model for password reset flows.
|
|
||||||
- JWT payload fields: `user_id`, `account_id`, `email`, `exp`, `iat`, `type`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Middleware step: `AccountContextMiddleware` determines `request.account` (session or JWT) and validates plan status; skips admin/auth routes.
|
|
||||||
- DRF auth step: API key/JWT/session/basic authenticators run in order, potentially setting `request.account` (API key/JWT) and `request.site` (API key).
|
|
||||||
- Viewsets then apply role/permission checks and tenant/site/sector filtering via base classes in `api/base.py`.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- All module viewsets rely on `request.user` and `request.account` set by the auth stack. Site-aware modules can read `request.site` when API key auth is used.
|
|
||||||
- Role helpers (`is_admin_or_developer`, `is_system_account_user`) influence filtering bypass in base viewsets.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- JWT lifetimes: access tokens default to 15 minutes; refresh tokens to 30 days (configurable in settings).
|
|
||||||
- Session users are refreshed on each request to pick up plan/account changes.
|
|
||||||
- Password reset tokens track expiry and usage via `expires_at` and `used` flags.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Middleware returns JSON errors for missing account or inactive plan and logs out session users in those cases.
|
|
||||||
- Invalid/expired JWTs cause the JWT authenticator to return `None`, allowing other auth methods; decoding errors raise `InvalidTokenError` in utilities.
|
|
||||||
- API key auth raises an auth failure when no active user is available for the resolved account.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- `request.account` is set early; base viewsets enforce account filtering unless user has admin/developer/system-account privileges.
|
|
||||||
- API key auth also sets `request.site` for integration contexts; site/sector filtering occurs in `SiteSectorModelViewSet`.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Active plan is required for access (middleware enforces). Credit debits/charges are handled in billing modules, not in the auth layer.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Token generation/validation is synchronous. Background tasks should receive explicit user/account identifiers in their payloads when invoked.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Authentication stack is ordered to give integration API keys precedence, then JWT for app clients, then session for browser-based flows.
|
|
||||||
- Tenant context must be established before view logic; do not move or remove `AccountContextMiddleware`.
|
|
||||||
- Expiry durations and JWT secrets are centrally configured in `settings.py`.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Use token helpers from `auth/utils.py` when issuing tokens; do not handcraft JWTs.
|
|
||||||
- Mount new auth-sensitive endpoints under existing routers and rely on DRF auth classes instead of custom header parsing.
|
|
||||||
- Ensure new features that require site context can work with API key auth by checking `request.site`.
|
|
||||||
- Keep plan enforcement in place; bypass only for admin/system routes when justified.
|
|
||||||
@@ -1,85 +0,0 @@
|
|||||||
# Data Flow Diagrams (Narrative)
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe end-to-end data movement through the system based on current routing, middleware, and model conventions. No diagrams are embedded; flows are explained textually.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Request routing: `backend/igny8_core/urls.py`
|
|
||||||
- Middleware: `backend/igny8_core/middleware/request_id.py`, `backend/igny8_core/auth/middleware.py`, `backend/igny8_core/middleware/resource_tracker.py`
|
|
||||||
- DRF base viewsets: `backend/igny8_core/api/base.py`
|
|
||||||
- Authentication classes: `backend/igny8_core/api/authentication.py`
|
|
||||||
- Tenancy models: `backend/igny8_core/auth/models.py`
|
|
||||||
- Celery configuration: `backend/igny8_core/settings.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Trace how HTTP requests are processed, tenant-scoped, authorized, and persisted.
|
|
||||||
- Show where async processing departs to Celery and where responses are shaped.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Incoming request → Django middleware stack:
|
|
||||||
- Security/WhiteNoise/CORS/session/common/CSRF/Django auth.
|
|
||||||
- `RequestIDMiddleware` assigns `request.request_id` and returns it in `X-Request-ID`.
|
|
||||||
- `AccountContextMiddleware` resolves user/account (session or JWT) and enforces active plan; sets `request.account`.
|
|
||||||
- `ResourceTrackingMiddleware` optionally tracks resource usage for admin/developer users when the `X-Debug-Resource-Tracking` header is true; adds `X-Resource-Tracking-ID`.
|
|
||||||
- URL dispatch via `urls.py` routes to module routers (auth, account, planner, writer, system, billing, automation, linker, optimizer, publisher, integration) under `/api/v1/*`.
|
|
||||||
- DRF viewset pipeline:
|
|
||||||
- Authentication classes (API key → JWT → session → basic) establish `request.user` (and optionally `request.account`/`request.site`).
|
|
||||||
- Base viewsets (`AccountModelViewSet`, `SiteSectorModelViewSet`) filter querysets by `account`/`site`/`sector` and attach `account` on create.
|
|
||||||
- Serializers handle validation; responses are wrapped by unified helpers to standardize success/error payloads and pagination.
|
|
||||||
- Persistence:
|
|
||||||
- Tenant-scoped models inherit `AccountBaseModel` or `SiteSectorBaseModel`; save hooks enforce account/site/sector alignment.
|
|
||||||
- Soft deletion is used where models implement `soft_delete`, respecting retention windows from account settings.
|
|
||||||
- Async/Background:
|
|
||||||
- Celery uses Redis broker/backend; tasks inherit JSON payloads and time limits from `settings.py`.
|
|
||||||
- Automation, AI, publishing, and billing tasks enqueue via Celery; results return through database/state updates, not synchronous responses.
|
|
||||||
- Response:
|
|
||||||
- Unified response wrappers ensure `success`, `data`/`error`, and request ID are present; paginated responses include `count/next/previous/results`.
|
|
||||||
- Throttling headers apply per-scope (as configured in `REST_FRAMEWORK` throttles).
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Tenancy bases: `AccountBaseModel`, `SiteSectorBaseModel`.
|
|
||||||
- Core entities: `Account`, `Plan`, `Site`, `Sector`, `User`, `SiteUserAccess`.
|
|
||||||
- Module-specific models follow the same tenancy bases (documented in module-specific files).
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
1) HTTP request hits middleware; IDs and tenant context are set.
|
|
||||||
2) DRF authentication authenticates and sets user/account/site.
|
|
||||||
3) Viewset filters data by tenant/site/sector and runs serializer validation.
|
|
||||||
4) DB operations persist data with enforced tenant alignment.
|
|
||||||
5) Optional Celery tasks are queued for long-running work.
|
|
||||||
6) Response returns unified JSON with request IDs and optional throttling/pagination headers.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Auth context set in middleware is consumed by all module viewsets for scoping.
|
|
||||||
- API key auth provides site context for integration/publisher flows.
|
|
||||||
- Celery configuration is shared by automation/AI/publishing/billing task modules.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Entity lifecycle changes (create/update/delete/soft-delete) flow through base viewsets and tenancy bases, ensuring account/site/sector consistency.
|
|
||||||
- Request lifecycle includes request ID creation, optional resource tracking, and unified response wrapping.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Middleware can short-circuit with JSON errors for missing account/plan.
|
|
||||||
- Viewset overrides wrap validation and server errors into unified responses; missing objects return 404 payloads.
|
|
||||||
- Throttling (scope-based) returns standard DRF throttle responses with headers.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All tenant-bound data flows require `request.account`; filtering and save hooks prevent cross-tenant access.
|
|
||||||
- Admin/developer/system-account users may bypass tenant filtering; system accounts are guarded against deletion.
|
|
||||||
- Site/sector alignment is validated on save for models inheriting `SiteSectorBaseModel`.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Plan activation is validated in middleware. Credit debits and billing workflows occur in billing modules (covered elsewhere) after tenant resolution.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Celery broker/backend configuration in `settings.py` governs async flow; tasks should include account/site identifiers to maintain scoping.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Request ID and resource tracking enable traceability and performance debugging.
|
|
||||||
- Middleware ordering ensures tenant context precedes view logic.
|
|
||||||
- Unified response format keeps clients consistent across modules.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Preserve middleware order; new middleware must not break request ID or tenant context.
|
|
||||||
- Ensure new viewsets inherit the base classes to pick up scoping and unified responses.
|
|
||||||
- When adding async tasks, include tenant/site identifiers and respect Celery limits from settings.
|
|
||||||
@@ -1,74 +0,0 @@
|
|||||||
# Permissions and Access Control
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Explain role-based permissions, account ownership rules, and how viewsets enforce access across the stack.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Role-based permissions: `backend/igny8_core/auth/permissions.py`
|
|
||||||
- User roles and helpers: `backend/igny8_core/auth/models.py`
|
|
||||||
- Account context middleware: `backend/igny8_core/auth/middleware.py`
|
|
||||||
- Base viewsets with override/bypass logic: `backend/igny8_core/api/base.py`
|
|
||||||
- DRF settings: `backend/igny8_core/settings.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Enforce role-based access (developer, owner, admin, editor, viewer, system_bot).
|
|
||||||
- Enforce tenant/site/sector scoping with selective overrides for privileged roles.
|
|
||||||
- Block access when accounts lack active plans.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Roles are stored on `User.role`; helper methods (`is_admin_or_developer`, `is_developer`, `is_system_account_user`, `has_role`) drive bypass logic.
|
|
||||||
- Permission classes:
|
|
||||||
- `IsOwnerOrAdmin`: allows authenticated users with roles owner/admin/developer or superuser.
|
|
||||||
- `IsEditorOrAbove`: allows editor/admin/owner/developer or superuser.
|
|
||||||
- `IsViewerOrAbove`: allows any authenticated user.
|
|
||||||
- `AccountPermission`: ensures the user is authenticated, not a system bot unless intended, and matches object account when present; system bots bypass account checks.
|
|
||||||
- Base viewsets:
|
|
||||||
- `AccountModelViewSet` filters by `account` unless `is_admin_or_developer` or `is_system_account_user` is true; create sets `account` automatically; delete protects system account slugs.
|
|
||||||
- `SiteSectorModelViewSet` extends filtering to `site`/`sector` when models have those fields.
|
|
||||||
- Middleware:
|
|
||||||
- `AccountContextMiddleware` denies requests when user lacks an account or active plan (403/402), logging out session users and returning JSON errors.
|
|
||||||
- System accounts (`aws-admin`, `default-account`, `default`) and system bots are privileged; system accounts are protected from deletion.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `User` with `role` and account FK.
|
|
||||||
- `Account` with `status` and `plan` used for gating.
|
|
||||||
- `SiteUserAccess` for per-site grants evaluated by `User.get_accessible_sites`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Middleware validates account/plan and sets `request.account`.
|
|
||||||
- DRF authentication sets `request.user`; permission classes on views (and base viewsets) check role/account match.
|
|
||||||
- Querysets are filtered by tenant/site/sector; privileged roles may bypass filtering.
|
|
||||||
- Soft-delete or delete operations respect system account protection.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- All module viewsets inherit base classes, so account/site/sector scoping and role-based bypass apply uniformly.
|
|
||||||
- Integration endpoints using API key auth rely on `request.site`/`request.account` set earlier.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Account status and plan activation affect access; changing these alters middleware decisions immediately due to per-request user refresh.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Middleware returns structured JSON errors for missing account or inactive plan.
|
|
||||||
- Base viewsets wrap permission and validation failures into unified error responses.
|
|
||||||
- Permission classes return standard DRF permission denials for unauthenticated/unauthorized users.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Default: filter data to `request.account`; site/sector models further constrain by site/sector.
|
|
||||||
- Overrides: developers/admins/system-account users can see across accounts; system bots can access any account/object; system accounts cannot be deleted.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Active plan required; enforced in middleware before permission checks.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Background tasks should be invoked with account/site identifiers to maintain the same isolation rules; role checks occur at request time, not within tasks.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Centralize role checks to avoid per-view duplication.
|
|
||||||
- Maintain middleware-first gating to prevent expensive processing before authorization.
|
|
||||||
- Preserve protection of system accounts and allow ops-level overrides without compromising tenant isolation.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Apply permission classes (`IsOwnerOrAdmin`, `IsEditorOrAbove`, etc.) on viewsets/endpoints as needed.
|
|
||||||
- Always inherit from `AccountModelViewSet`/`SiteSectorModelViewSet` for scoped data access.
|
|
||||||
- Avoid bypassing middleware plan checks; if admin-only, use explicit admin routes instead.
|
|
||||||
- When adding new privileged behaviors, reuse `User` role helpers and system-account detection.
|
|
||||||
@@ -1,91 +0,0 @@
|
|||||||
# Environment Variables
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
List environment variables that influence runtime behavior, taken directly from `settings.py` and related auth/Celery configuration.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Primary definitions and defaults: `backend/igny8_core/settings.py`
|
|
||||||
- JWT helpers: `backend/igny8_core/auth/utils.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Configure secrets, debug flags, DB connections, CORS, auth, Celery, and billing providers without code changes.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Core flags:
|
|
||||||
- `SECRET_KEY` (required for production)
|
|
||||||
- `DEBUG` (enables debug when set to true)
|
|
||||||
- `IGNY8_DEBUG_THROTTLE` (bypass rate limiting when true; defaults to `DEBUG`)
|
|
||||||
- `IGNY8_USE_UNIFIED_EXCEPTION_HANDLER` (controls custom DRF exception handler; enabled by default)
|
|
||||||
- `USE_SITE_BUILDER_REFACTOR` (feature flag for site builder)
|
|
||||||
- `USE_SECURE_COOKIES` (sets session/CSRF secure cookies)
|
|
||||||
- `USE_SECURE_PROXY_HEADER` (enables `SECURE_PROXY_SSL_HEADER`)
|
|
||||||
- `USE_X_FORWARDED_HOST` is disabled by default; no env provided.
|
|
||||||
- Database selection (PostgreSQL default with SQLite fallbacks):
|
|
||||||
- `DATABASE_URL` (parsed for engine/user/pass/host/port/name; supports postgres or sqlite)
|
|
||||||
- `DB_ENGINE` (forces sqlite when set to sqlite/sqlite3)
|
|
||||||
- `DJANGO_FORCE_POSTGRES` (force Postgres even in debug)
|
|
||||||
- `DB_NAME`, `DB_USER`, `DB_PASSWORD`, `DB_HOST`, `DB_PORT` (used when `DATABASE_URL` not provided)
|
|
||||||
- `USE_SQLITE` and `SQLITE_NAME` (explicit sqlite override)
|
|
||||||
- CORS/hosts:
|
|
||||||
- CORS origins are hardcoded; no env toggle beyond `DEBUG`. Trusted CSRF origins are static in settings.
|
|
||||||
- Auth/JWT:
|
|
||||||
- `JWT_SECRET_KEY` (defaults to `SECRET_KEY`)
|
|
||||||
- `JWT_ALGORITHM` (defaults to HS256)
|
|
||||||
- `JWT_ACCESS_TOKEN_EXPIRY` (timedelta in settings; default 15 minutes)
|
|
||||||
- `JWT_REFRESH_TOKEN_EXPIRY` (default 30 days)
|
|
||||||
- Celery/Redis:
|
|
||||||
- `CELERY_BROKER_URL` (defaults to `redis://{REDIS_HOST}:{REDIS_PORT}/0`)
|
|
||||||
- `CELERY_RESULT_BACKEND` (defaults to same as broker)
|
|
||||||
- `REDIS_HOST`, `REDIS_PORT` (defaults redis:6379)
|
|
||||||
- `REDIS_SENTINEL_ENABLED` (when true, adds sentinel backend options)
|
|
||||||
- `REDIS_SSL_ENABLED` (enables Redis backend SSL)
|
|
||||||
- Feature/config paths:
|
|
||||||
- `PUBLISH_SYNC_LOG_DIR` is derived; no env override.
|
|
||||||
- Payments:
|
|
||||||
- `STRIPE_PUBLIC_KEY`, `STRIPE_SECRET_KEY`, `STRIPE_WEBHOOK_SECRET`
|
|
||||||
- `PAYPAL_CLIENT_ID`, `PAYPAL_CLIENT_SECRET`, `PAYPAL_API_BASE` (defaults to sandbox base)
|
|
||||||
- Rate limiting and throttles:
|
|
||||||
- Scopes are defined in settings; toggling is via `IGNY8_DEBUG_THROTTLE`.
|
|
||||||
- Security/Cookies:
|
|
||||||
- `USE_SECURE_COOKIES`, `USE_SECURE_PROXY_HEADER` control secure cookie and proxy behavior.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- None; variables configure runtime services used by auth, database, Celery, and billing.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Django reads env vars at import of `settings.py`; defaults apply when unset.
|
|
||||||
- Token helpers read JWT secrets/algorithms/expiries for generation and validation.
|
|
||||||
- Celery settings are consumed by workers/beat at startup.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Auth stack uses JWT settings and secrets.
|
|
||||||
- Account middleware uses plan validation; DB settings drive tenancy storage.
|
|
||||||
- Celery settings affect AI/automation/publishing/billing tasks.
|
|
||||||
- Stripe/PayPal keys are consumed by billing modules.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Changes to env vars take effect on process restart; token expiry/secret changes invalidate existing tokens accordingly.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Missing/invalid DB env falls back to defaults (SQLite in debug unless forced).
|
|
||||||
- Missing JWT secret falls back to `SECRET_KEY`; an absent secret raises errors during decode if unset.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Env vars do not alter tenancy logic; they configure infrastructure supporting it.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Stripe/PayPal keys must be present for payment webhooks/operations; plan enforcement itself is not env-driven.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Celery broker/backend URLs and Redis options come from env; task timeouts and serializer are fixed in settings.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Favor explicit env configuration for production (secrets, DB, Redis, Stripe/PayPal).
|
|
||||||
- Keep debug-only flags (`DEBUG`, `IGNY8_DEBUG_THROTTLE`) off in production.
|
|
||||||
- Use `DATABASE_URL` for portability; fallback logic supports sqlite for local dev.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Set required secrets (`SECRET_KEY`, `JWT_SECRET_KEY`, Stripe/PayPal keys) before deploying.
|
|
||||||
- Choose DB via `DATABASE_URL` or explicit `DB_*` vars; avoid sqlite in production unless intentional.
|
|
||||||
- Configure Redis URLs for Celery in non-dev environments.
|
|
||||||
- Restart services after changing env vars to apply new configuration.
|
|
||||||
@@ -1,104 +0,0 @@
|
|||||||
# Multi-Tenancy & Access Reference (Current State)
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Authoritative map of tenant isolation, role access, and payment/API-key handling across the stack. Built from code as of Dec 2025.
|
|
||||||
|
|
||||||
## Core Enforcement Points (backend)
|
|
||||||
- Middleware:
|
|
||||||
- `backend/igny8_core/auth/middleware.py` (`AccountContextMiddleware`, ~L1-L220): resolves `request.account` from JWT/API key; blocks inactive/suspended accounts.
|
|
||||||
- `backend/igny8_core/middleware/request_id.py` (~L1-L70): request ID (not tenancy).
|
|
||||||
- `backend/igny8_core/middleware/resource_tracker.py` (~L1-L170): metrics (not tenancy).
|
|
||||||
- Base viewsets:
|
|
||||||
- `backend/igny8_core/api/base.py` (`AccountModelViewSet`, ~L1-L240): filters by `request.account`; admin/developer/system overrides; sets account on create.
|
|
||||||
- `backend/igny8_core/api/base.py` (`SiteSectorModelViewSet`, ~L238-L430): additionally filters by site/sector and user’s accessible sites (SiteUserAccess) unless admin/developer/system.
|
|
||||||
- Permissions:
|
|
||||||
- `backend/igny8_core/api/permissions.py`:
|
|
||||||
- `IsAuthenticatedAndActive`, `HasTenantAccess` (default in settings).
|
|
||||||
- `IsViewerOrAbove`, `IsEditorOrAbove`, `IsAdminOrOwner`.
|
|
||||||
- `IsSystemAccountOrDeveloper` (system/admin for integrations).
|
|
||||||
- Module-specific permissions also appear in `backend/igny8_core/auth/permissions.py` (legacy IsOwnerOrAdmin, IsEditorOrAbove, IsViewerOrAbove, AccountPermission).
|
|
||||||
- Settings defaults:
|
|
||||||
- `backend/igny8_core/settings.py` REST_FRAMEWORK `DEFAULT_PERMISSION_CLASSES` = `IsAuthenticatedAndActive` + `HasTenantAccess`.
|
|
||||||
- Auth order: APIKeyAuthentication → JWTAuthentication → CSRFExemptSessionAuthentication → BasicAuth.
|
|
||||||
- Throttling: `DebugScopedRateThrottle` bypasses throttles for authenticated users/system/debug.
|
|
||||||
- Models with enforced account/site/sector:
|
|
||||||
- Base models `AccountBaseModel`, `SiteSectorBaseModel` in `backend/igny8_core/auth/models.py` (top of file).
|
|
||||||
|
|
||||||
## Flow (text flowchart)
|
|
||||||
```
|
|
||||||
Request
|
|
||||||
-> Middleware: AccountContextMiddleware sets request.account (JWT/API key), validates account status/plan
|
|
||||||
-> DRF Auth: APIKey/JWT/Session
|
|
||||||
-> Permissions: IsAuthenticatedAndActive + HasTenantAccess (+ role-specific)
|
|
||||||
-> ViewSet:
|
|
||||||
AccountModelViewSet filters by account
|
|
||||||
SiteSectorModelViewSet filters by account + site/sector + SiteUserAccess
|
|
||||||
-> Action-specific role checks (IsEditorOrAbove, IsAdminOrOwner, IsSystemAccountOrDeveloper)
|
|
||||||
-> Business logic (services) + credit checks (billing)
|
|
||||||
-> Response
|
|
||||||
```
|
|
||||||
|
|
||||||
## Module Access (backend ViewSets & guards)
|
|
||||||
- Accounts/Users/Plans/Subscriptions:
|
|
||||||
- `auth/views.py`: `UsersViewSet`, `AccountsViewSet`, `SubscriptionsViewSet`, `SiteUserAccessViewSet` (account-scoped via AccountModelViewSet + role guards).
|
|
||||||
- Roles: owner/admin (or developer/system) can manage; others limited to self (UsersViewSet get_queryset).
|
|
||||||
- Sites/Sectors:
|
|
||||||
- `auth/views.py` (`SiteViewSet`, `Sector` actions): SiteSectorModelViewSet enforces account + site/sector + SiteUserAccess; public slug read is AllowAny for active site slug only.
|
|
||||||
- Planner:
|
|
||||||
- `modules/planner/views.py` (KeywordViewSet, ClusterViewSet, ContentIdeasViewSet) inherit SiteSectorModelViewSet; require site_id/sector_id; role: typically editor+ for writes.
|
|
||||||
- Writer:
|
|
||||||
- `modules/writer/views.py` (TasksViewSet, ContentViewSet, ImagesViewSet, ContentTaxonomyViewSet) inherit SiteSectorModelViewSet; site/sector scoping; editor+ for writes.
|
|
||||||
- Automation:
|
|
||||||
- `business/automation/views.py` (AutomationViewSet) inherits AccountModelViewSet/SiteSectorModelViewSet patterns; requires site_id for run/config; role: editor+ for mutate.
|
|
||||||
- System settings (non-integrations):
|
|
||||||
- `modules/system/views.py` / `settings_views.py`: AccountModelViewSet; role usually admin/owner; authenticated + tenant required.
|
|
||||||
- Integrations (OpenAI/Runware API keys):
|
|
||||||
- `modules/system/integration_views.py`: guarded by `IsSystemAccountOrDeveloper` (system account or developer only); tenant-scoped but effectively system-only for keys.
|
|
||||||
- Billing:
|
|
||||||
- `modules/billing/views.py`: AccountModelViewSet; `IsAdminOrOwner` for credit transactions/payment methods; balance/usage requires auth + tenant.
|
|
||||||
- Payments/Payment Methods:
|
|
||||||
- Payment methods: `AccountPaymentMethodViewSet` account-scoped; IsAuthenticated; default selection per account; admin/owner should manage.
|
|
||||||
- Payments: `PaymentViewSet` account-scoped; IsAuthenticated; list/available_methods/manual payment for current account only.
|
|
||||||
|
|
||||||
## Frontend Guards
|
|
||||||
- Route protection: `ProtectedRoute` (auth required, checks account/plan/payment methods), `ModuleGuard` (module enabled), `AdminGuard` (integration/admin pages only for system account or developer).
|
|
||||||
- Sidebar hides Integration for non-system/developer; admin section shown only for system/developer.
|
|
||||||
|
|
||||||
## AI Key Resolution
|
|
||||||
- `ai/ai_core.py` `_load_account_settings`: tries tenant IntegrationSettings → system account IntegrationSettings (`aws-admin`/`default-account`/`default`) → Django settings (`OPENAI_API_KEY`, `RUNWARE_API_KEY`). All users run AI with shared keys if tenant keys absent.
|
|
||||||
|
|
||||||
## Throttling
|
|
||||||
- `api/throttles.py` `DebugScopedRateThrottle`: bypass for authenticated users/system/debug; per-scope rates in `settings.py`. Prevents 429s for normal users.
|
|
||||||
|
|
||||||
## Payment / Billing Workflow (happy path)
|
|
||||||
1) User authenticates (JWT) → request.account set.
|
|
||||||
2) Payment methods (account-scoped) fetched via `/v1/billing/payment-methods/available/`; admin/owner can CRUD `/v1/billing/payment-methods/`.
|
|
||||||
3) Invoices/Payments via billing endpoints (account-scoped; admin/owner).
|
|
||||||
4) Credits used via CreditService on AI/automation calls (backend).
|
|
||||||
|
|
||||||
## Access Summary by Role (runtime enforcement)
|
|
||||||
- Viewer: read-only where viewsets allow `IsViewerOrAbove`; no writes.
|
|
||||||
- Editor: can write planner/writer/automation; cannot manage billing/integration.
|
|
||||||
- Admin/Owner: manage account/team/billing/payment methods; full module writes.
|
|
||||||
- Developer/System account: cross-tenant overrides in some base filters; integration settings and admin menus.
|
|
||||||
|
|
||||||
## Key Files (with line bands)
|
|
||||||
- Middleware: `auth/middleware.py` (~L1-220)
|
|
||||||
- Base viewsets: `api/base.py` (~L1-430)
|
|
||||||
- Permissions: `api/permissions.py` (~L1-200), `auth/permissions.py` (~L1-120)
|
|
||||||
- Settings (REST/Throttle): `settings.py` (REST_FRAMEWORK block, ~L200-360)
|
|
||||||
- AI core key loading: `ai/ai_core.py` (~L1-120)
|
|
||||||
- Integration settings views: `modules/system/integration_views.py` (~L1-300 main guards; actions throughout)
|
|
||||||
- Planner views: `modules/planner/views.py` (all ViewSets inherit SiteSectorModelViewSet)
|
|
||||||
- Writer views: `modules/writer/views.py`
|
|
||||||
- Automation: `business/automation/views.py`, `services/automation_service.py`
|
|
||||||
- Billing: `modules/billing/views.py`, `business/billing/services/credit_service.py`
|
|
||||||
- Payment methods: `modules/billing/views.py` AccountPaymentMethodViewSet
|
|
||||||
- Frontend guards: `src/components/auth/ProtectedRoute.tsx`, `src/components/auth/AdminGuard.tsx`, `src/components/common/ModuleGuard.tsx`
|
|
||||||
- Sidebar gating: `src/layout/AppSidebar.tsx`
|
|
||||||
|
|
||||||
## Open Items / Risks
|
|
||||||
- Ensure public endpoints explicitly override default permissions (e.g., auth register/login, site slug read).
|
|
||||||
- Validate all viewsets still inherit AccountModelViewSet/SiteSectorModelViewSet after future changes.
|
|
||||||
- Add automated tests for cross-tenant denial, role gates, plan limits, and integration access.***
|
|
||||||
|
|
||||||
@@ -1,82 +0,0 @@
|
|||||||
# Backend Architecture
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Explain how the backend is structured, wired, and executed across Django/DRF, middleware, routing, async processing, and logging.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Settings and app wiring: `backend/igny8_core/settings.py`
|
|
||||||
- URL routing: `backend/igny8_core/urls.py`
|
|
||||||
- Middleware: `backend/igny8_core/middleware/request_id.py`, `backend/igny8_core/auth/middleware.py`, `backend/igny8_core/middleware/resource_tracker.py`
|
|
||||||
- Auth stack: `backend/igny8_core/api/authentication.py`, `backend/igny8_core/auth/utils.py`
|
|
||||||
- Base viewsets and unified responses: `backend/igny8_core/api/base.py`
|
|
||||||
- Domain models: `backend/igny8_core/auth/models.py` plus `backend/igny8_core/business/*/models.py`
|
|
||||||
- Async/Celery config: `backend/igny8_core/settings.py`
|
|
||||||
- Logging: `backend/igny8_core/settings.py` (publish/webhook logs), automation logging (`backend/igny8_core/business/automation/services/automation_logger.py`)
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Django/DRF API surface under `/api/v1/*` for planner, writer, system, billing, automation, linker, optimizer, publisher, and integration modules.
|
|
||||||
- Middleware establishes request IDs, tenant context, and optional resource tracking before views run.
|
|
||||||
- DRF configuration standardizes pagination, filtering, authentication, throttling, and exception handling.
|
|
||||||
- Celery + Redis provide async execution for AI, automation, publishing, and other long-running tasks.
|
|
||||||
- Logging and CORS/security settings are centralized in settings.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- Apps registered in `INSTALLED_APPS` include auth, AI framework, planner, writer, system, billing, automation, optimization, publishing, integration, linker, optimizer, publisher. Custom admin config is loaded first.
|
|
||||||
- Middleware order: security → WhiteNoise → CORS → session/common/CSRF → Django auth → `RequestIDMiddleware` → `AccountContextMiddleware` (tenant + plan enforcement) → `ResourceTrackingMiddleware` (opt-in for admin/developer with header) → messages → clickjacking.
|
|
||||||
- Routing (`urls.py`): admin plus CSV helpers for industries/seed keywords, then `/api/v1/` routers for auth, account, planner, writer, system, billing (user/admin), automation, linker, optimizer, publisher, integration. OpenAPI served at `/api/docs` and `/api/redoc`.
|
|
||||||
- DRF defaults: unified exception handler (unless env disables), custom pagination, filtering/search/ordering, auth stack (API key → JWT → CSRF-exempt session → basic), scoped throttling per operation class, drf-spectacular schema generation with tag ordering.
|
|
||||||
- CORS: IGNY8 domains and local dev ports allowed; credentials enabled; request/resource tracking headers exposed.
|
|
||||||
- Celery: Redis broker/result; JSON serialization; task/soft time limits; single-prefetch; sentinel/SSL toggles via env.
|
|
||||||
- Logging: console plus rotating files for publish/sync, WordPress API, webhooks; request IDs from middleware surface in responses; automation has dedicated file-based logger per run.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- Tenancy/identity: `Account`, `Plan`, `Subscription`, `Site`, `Sector`, `User`, `SiteUserAccess`, `PasswordResetToken` (in auth models).
|
|
||||||
- Domain models across planner, writer, billing, automation, publishing, integration, optimization (see domain models doc).
|
|
||||||
- Middleware uses request-scoped `request.account`, `request.site`, and `request.request_id`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
1) Request enters middleware stack; IDs and tenant context are set; plan is verified.
|
|
||||||
2) DRF auth classes authenticate user/api-key/JWT/session.
|
|
||||||
3) Base viewsets filter by tenant/site/sector and handle unified responses.
|
|
||||||
4) Serializers validate; database writes enforce tenant alignment via model bases.
|
|
||||||
5) Optional Celery tasks are enqueued for long-running work; responses remain synchronous JSON with pagination/throttle headers when applicable.
|
|
||||||
6) Logging writes to configured handlers; request/resource IDs are added to responses.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Tenant context from middleware is consumed by all module viewsets.
|
|
||||||
- AI and automation invoke Celery tasks and AI functions; billing services deduct credits used by automation/AI flows.
|
|
||||||
- Integration/publishing modules rely on API key auth to set `request.site` for WordPress and other platforms.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Account/plan validity is checked per request; system accounts are protected from deletion.
|
|
||||||
- Soft-delete is available on many models via `SoftDeletableModel`.
|
|
||||||
- Request lifecycle includes request ID creation, optional resource tracking, and unified response formatting.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Custom DRF exception handler wraps errors with `success=false`.
|
|
||||||
- `AccountContextMiddleware` blocks requests lacking account or active plan (403/402 JSON).
|
|
||||||
- Base viewsets wrap validation/404/500 errors into unified payloads.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Request-level `account` (and `site`/`sector` where applicable) is injected by middleware/auth; base viewsets enforce filtering.
|
|
||||||
- Admin/developer/system-account users bypass tenant filtering; system accounts are guarded from deletion.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Billing env keys configured in settings; plan enforcement occurs in middleware; credit debits handled by billing services during operations.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Celery worker/beat use Redis URLs from settings; task limits and JSON serialization enforced.
|
|
||||||
- Automation, AI, publishing, and optimization tasks run async; automation logger writes per-run files.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Middleware-first tenant enforcement ensures isolation.
|
|
||||||
- Unified API standards (responses, throttles, schema) keep clients consistent.
|
|
||||||
- Celery offloads expensive AI/automation/publishing work with bounded execution.
|
|
||||||
- Logging and request IDs aid observability; resource tracking is opt-in for admins.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Register new apps in `INSTALLED_APPS` and route them under `/api/v1/*`.
|
|
||||||
- Use existing middleware order; add new middleware only if it does not break account/request ID handling.
|
|
||||||
- Inherit from base viewsets for scoping and response consistency.
|
|
||||||
- Use Celery for long-running tasks; respect Redis/task time-limit settings.
|
|
||||||
- Keep env vars (DB, JWT, Redis, Stripe/PayPal) set per `settings.py`; avoid hardcoded secrets.
|
|
||||||
@@ -1,110 +0,0 @@
|
|||||||
# Domain Models
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe the key backend models, their responsibilities, constraints, and tenancy rules, grounded in current implementations.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Tenancy bases and identity: `backend/igny8_core/auth/models.py`
|
|
||||||
- Planner models: `backend/igny8_core/business/planning/models.py`
|
|
||||||
- Writer/content models: `backend/igny8_core/business/content/models.py`
|
|
||||||
- Automation models: `backend/igny8_core/business/automation/models.py`
|
|
||||||
- Billing models: `backend/igny8_core/business/billing/models.py`
|
|
||||||
- Integration models: `backend/igny8_core/business/integration/models.py`
|
|
||||||
- Publishing models: `backend/igny8_core/business/publishing/models.py`
|
|
||||||
- Optimization models: `backend/igny8_core/business/optimization/models.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Provide tenant-scoped storage for planner (keywords/clusters/ideas), writer (tasks/content/taxonomies/images/attributes), automation runs/config, billing (credits, invoices, payments), integration metadata, publishing records, and optimization tasks.
|
|
||||||
- Enforce account/site/sector alignment via base classes and save-time validation.
|
|
||||||
- Track external platform links (WordPress/Shopify/custom), credit usage, and publishing/optimization state.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
### Tenancy Bases (auth/models.py)
|
|
||||||
- `AccountBaseModel`: adds `account`, timestamps, and indexes; all tenant models inherit.
|
|
||||||
- `SiteSectorBaseModel`: extends with `site` and `sector`; save enforces site → account alignment and sector belonging to the same site; raises validation errors if mismatched.
|
|
||||||
|
|
||||||
### Planner (business/planning/models.py)
|
|
||||||
- `Clusters`: tenant/site/sector-scoped keyword group; tracks counts, volume, mapped pages, status (`new/mapped`), disable flag; unique per site/sector by name; soft-deletable.
|
|
||||||
- `Keywords`: tenant/site/sector-scoped keyword tied to a global `SeedKeyword`; optional overrides for volume/difficulty/attributes; optional cluster link (same sector enforced); validation ensures seed keyword industry/sector matches site/sector; status (`new/mapped`), disable flag; soft-deletable.
|
|
||||||
- `ContentIdeas`: ideas tied to clusters and optional keywords; tracks status (`new/queued/completed`), content type/structure, estimated word count; soft-deletable.
|
|
||||||
|
|
||||||
### Writer / Content (business/content/models.py)
|
|
||||||
- `Tasks`: queue items for content generation; tied to cluster (required) and optional idea/taxonomy; content type/structure, keywords text, target word count, status (`queued/completed`); soft-deletable.
|
|
||||||
- `Content`: generated or imported content; stores HTML, word count, SEO fields, cluster link, content type/structure, taxonomy M2M, external IDs/URLs/metadata, sync status, source (`igny8/wordpress`), and status (`draft/review/published`); soft-deletable.
|
|
||||||
- `ContentTaxonomy`: simplified taxonomy (category/tag) with external taxonomy/ID, sync status, description, count, metadata; unique per site by slug/type and by external ID/taxonomy.
|
|
||||||
- `Images`: images linked to content or task; auto-populates account/site/sector from the linked object; tracks type, URL/path, prompt, status, position; soft-deletable.
|
|
||||||
- `ContentClusterMap`: maps content/tasks to clusters with role (`hub/supporting/attribute`) and source (`blueprint/manual/import`); auto-populates tenant context from linked content/task; unique per content+cluster+role.
|
|
||||||
- `ContentAttribute` (alias `ContentAttributeMap`): tenant/site/sector-scoped attributes for content/task/cluster; typed (`product_spec/service_modifier/semantic_facet`), with optional external IDs, sources, and metadata; auto-populates tenant context from linked content/task.
|
|
||||||
|
|
||||||
### Automation (business/automation/models.py)
|
|
||||||
- `AutomationConfig`: per-site config with enable flag, frequency (`daily/weekly/monthly`), scheduled time, batch sizes per stage, within/between-stage delays, and next/last run timestamps.
|
|
||||||
- `AutomationRun`: tracks each run with trigger (`manual/scheduled`), status (`running/paused/cancelled/completed/failed`), current stage, pause/cancel timestamps, start/end, total credits used, per-stage JSON results, and optional error message.
|
|
||||||
|
|
||||||
### Billing (business/billing/models.py)
|
|
||||||
- `CreditTransaction`: ledger of credit changes (purchase/subscription/refund/deduction/adjustment) with balance-after and metadata.
|
|
||||||
- `CreditUsageLog`: detailed AI usage log with operation type (clustering/idea/content/image/reparse/legacy names), credits used, optional cost/model/tokens, related object references, and metadata.
|
|
||||||
- `CreditCostConfig`: admin-configurable credit costs per operation with unit (per request/words/items/images), display metadata, active flag, audit fields, and previous cost tracking.
|
|
||||||
- `Invoice`: tenant invoice with amounts, status (`draft/pending/paid/void/uncollectible`), dates, subscription link, line items JSON, payment metadata, Stripe IDs, notes; helper properties mirror legacy fields.
|
|
||||||
- `Payment`: payment records per invoice with status lifecycle (pending/processing/succeeded/completed/failed/refunded/cancelled), method (Stripe/PayPal/bank/local wallet/manual), provider references, manual notes/approval fields, failure reason, timestamps, metadata.
|
|
||||||
- `CreditPackage`: one-time credit bundles with price, discount, Stripe/PayPal IDs, active/featured flags, description/features, sort order.
|
|
||||||
- `PaymentMethodConfig`: per-country payment-method availability and display/instruction fields; includes bank/local wallet metadata; unique per country+method.
|
|
||||||
- `AccountPaymentMethod`: account-level payment metadata (non-sensitive) with type, display name, default/enabled/verified flags, country code, instructions, metadata; unique per account+display name.
|
|
||||||
|
|
||||||
### Integration (business/integration/models.py)
|
|
||||||
- `SiteIntegration`: tenant/site-specific integration config with platform (`wordpress/shopify/custom`), platform type (`cms/ecommerce/custom_api`), config JSON, credentials JSON, active/sync flags, sync status, last sync/error, timestamps; unique per site+platform.
|
|
||||||
- `SyncEvent`: event log per integration/site with event/action types, success flag, optional content/external IDs, details JSON, error, duration, and timestamps; indexed for debugging feeds.
|
|
||||||
|
|
||||||
### Publishing (business/publishing/models.py)
|
|
||||||
- `PublishingRecord`: tracks content publishing to destinations (wordpress/sites/shopify) with destination IDs/URLs, status (`pending/publishing/published/failed`), timestamps, errors, metadata; site/sector scoped via base.
|
|
||||||
- `DeploymentRecord`: tracks site deployments (sites renderer) with version/deployed_version, status (`pending/deploying/deployed/failed/rolled_back`), deployment URL, error, metadata, timestamps; site/sector scoped.
|
|
||||||
|
|
||||||
### Optimization (business/optimization/models.py)
|
|
||||||
- `OptimizationTask`: content optimization runs with before/after scores and HTML, status (`pending/running/completed/failed`), credits used, metadata; auto-sets account from content; tenant scoped.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Tenant context is inherited from base models; many save methods propagate account/site/sector from related entities (e.g., Images, ContentClusterMap, ContentAttribute).
|
|
||||||
- Planner → Writer linkage: Keywords and Clusters feed ContentIdeas; Tasks reference clusters/ideas; Content references clusters and taxonomies; Images/Attributes link to Tasks/Content.
|
|
||||||
- Automation runs reference planner/writer models and record per-stage outputs; configs control batching/delays.
|
|
||||||
- Billing logs and cost configs govern credit debits triggered by services (see services doc).
|
|
||||||
- Integration/publishing models bind site integrations and publishing deployments to site-scoped content.
|
|
||||||
- Optimization tasks attach to content and capture before/after artifacts.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Planner and writer share clusters/ideas/tasks/content relationships.
|
|
||||||
- Billing models are invoked by services during AI/automation/image/content operations.
|
|
||||||
- Integration events reference content IDs and external IDs for sync traces.
|
|
||||||
- Publishing records reference writer content; deployment records reference sites.
|
|
||||||
- Optimization tasks reference writer content and can influence publishing readiness downstream.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Soft-delete is available on planner keywords/clusters/ideas and writer tasks/content/images via `SoftDeletableModel`.
|
|
||||||
- Status fields track lifecycle: planner (`new/mapped/queued/completed`), writer tasks (`queued/completed`), content (`draft/review/published`), automation (`running/paused/cancelled/completed/failed`), publishing/deployment statuses, payment/invoice statuses, optimization statuses.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Save-time validation in `SiteSectorBaseModel` and `Keywords` ensures tenant/site/sector alignment and industry/sector matching.
|
|
||||||
- Unique constraints prevent duplicate clusters/keywords per site/sector and overlapping taxonomies/external IDs.
|
|
||||||
- Automation runs store error messages and partial stage results; publishing/deployment records store error text.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All models shown are tenant scoped via `AccountBaseModel` or `SiteSectorBaseModel`; save hooks propagate context from related objects where needed.
|
|
||||||
- Privileged roles can bypass filtering at the viewset layer, but persisted records retain account/site/sector ownership.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Credits reside on `Account`; transactions/usage logs record debits/credits; cost configs define per-operation pricing.
|
|
||||||
- Invoices/payments/credit packages configure monetary flows; payment methods can be toggled per country or per account.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Automation configs drive scheduled runs; automation runs record stage outputs and timing.
|
|
||||||
- Publishing/optimization tasks may be executed async via services/Celery (see services doc).
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Tenant isolation is encoded at the model layer via base classes and validation, ensuring downstream services inherit scoping.
|
|
||||||
- Cross-module links (clusters ↔ tasks ↔ content ↔ publishing/optimization) keep content lifecycle traceable.
|
|
||||||
- Billing and integration models include metadata fields to avoid schema churn while capturing provider-specific details.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Inherit new tenant models from `AccountBaseModel` or `SiteSectorBaseModel` to enforce scoping automatically.
|
|
||||||
- Validate cross-entity alignment (site/sector/industry) when relating planner and writer records.
|
|
||||||
- Use existing status fields/choices when extending lifecycles; preserve unique constraints when adding fields.
|
|
||||||
- When integrating new providers, extend or add models parallel to `SiteIntegration`/`SyncEvent` and keep platform-specific data in JSON fields.
|
|
||||||
|
|
||||||
@@ -1,96 +0,0 @@
|
|||||||
# Services and Modules
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Describe the backend service layer and module wiring that orchestrate domain models, AI/automation, billing, integration, and publishing.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Automation services: `backend/igny8_core/business/automation/services/automation_service.py`, `automation_logger.py`
|
|
||||||
- Billing services: `backend/igny8_core/business/billing/services/credit_service.py`, `invoice_service.py`, `payment_service.py`
|
|
||||||
- Integration service: `backend/igny8_core/business/integration/services/integration_service.py`
|
|
||||||
- Additional service directories (see module-specific docs for details):
|
|
||||||
- Planner: `backend/igny8_core/business/planning/services/*`
|
|
||||||
- Content/Writer: `backend/igny8_core/business/content/services/*`
|
|
||||||
- Automation tasks: `backend/igny8_core/business/automation/tasks.py`
|
|
||||||
- Integration sync: `backend/igny8_core/business/integration/services/*`
|
|
||||||
- Publishing: `backend/igny8_core/business/publishing/services/*`
|
|
||||||
- Optimization: `backend/igny8_core/business/optimization/services/*`
|
|
||||||
- Linking: `backend/igny8_core/business/linking/services/*`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Orchestrate multi-stage automation that chains planner and writer operations using AI and credits.
|
|
||||||
- Manage credit pricing, balance checks, deductions, and ledger logging.
|
|
||||||
- Generate invoices and handle billing documents.
|
|
||||||
- Create, update, test, and list site integrations for external platforms.
|
|
||||||
- Provide domain-specific service hooks for planner, writer, publishing, linking, and optimization flows (behavior documented in module-specific files).
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
### Automation
|
|
||||||
- `AutomationService` enforces single concurrent run per site via cache lock, estimates required credits, checks account balance (with buffer), and creates an `AutomationRun`. It sequences stages (keywords→clusters, clusters→ideas, ideas→tasks/content, image prompt generation, image generation queue) using AI functions (`AutoClusterFunction`, `GenerateIdeasFunction`, `GenerateContentFunction`, `GenerateImagePromptsFunction`) through `AIEngine`, and the Celery image queue (`process_image_generation_queue`). It supports pause/cancel checks mid-stage, records partial progress, and advances `current_stage` with per-stage result JSON and credit tallies. Batch sizes and delays respect `AutomationConfig`.
|
|
||||||
- `AutomationLogger` creates per-run directories (and optional shared mirrors), generates run IDs, writes main/stage logs, mirrors to shared folders when configured, and emits structured JSONL trace events for run start, progress, completion, and errors.
|
|
||||||
|
|
||||||
### Billing
|
|
||||||
- `CreditService` computes credit cost by first consulting `CreditCostConfig` (unit-aware for words/items/images) and falling back to constants. It checks balances, deducts credits atomically, updates account balance, writes `CreditTransaction`, and logs usage in `CreditUsageLog` with optional model/token metadata. It can also add credits and provides legacy check helpers.
|
|
||||||
- `InvoiceService` generates invoice numbers per account/month, creates invoices for subscriptions or credit packages (adding line items and computing totals), supports custom invoices, and marks invoices paid or void. PDF generation is currently a placeholder.
|
|
||||||
- `PaymentService` (see file) processes payments against invoices and updates statuses; details are documented in the module file.
|
|
||||||
|
|
||||||
### Integration
|
|
||||||
- `IntegrationService` creates, updates, deletes, fetches, and lists `SiteIntegration` records, setting account/site automatically. It can test connections per platform (WordPress, Shopify) and delegates to platform-specific test helpers inside the service; unimplemented platforms raise `NotImplementedError`. Credentials are set via `set_credentials`, which currently stores JSON as-is.
|
|
||||||
- Additional sync services (`content_sync_service.py`, `sync_service.py`, `sync_metadata_service.py`, `sync_health_service.py`) coordinate publish/sync flows and health checks; see module docs for specifics.
|
|
||||||
|
|
||||||
### Other Service Areas (structure)
|
|
||||||
- Planner services (`clustering_service.py`, `ideas_service.py`) handle clustering/idea logic.
|
|
||||||
- Content services (`content_generation_service.py`, `content_pipeline_service.py`, `metadata_mapping_service.py`, `validation_service.py`) manage content generation, pipelines, metadata mapping, and validation.
|
|
||||||
- Publishing services (`publisher_service.py`, `deployment_service.py`, `adapters/`) manage publishing/deployment flows and destination adapters.
|
|
||||||
- Optimization services (`analyzer.py`, `optimizer_service.py`) analyze and optimize content.
|
|
||||||
- Linking services (`candidate_engine.py`, `injection_engine.py`, `linker_service.py`) prepare and apply link suggestions.
|
|
||||||
- Automation tasks (`business/automation/tasks.py`) provide Celery entrypoints for automation runs.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Automation: `AutomationService.start_automation` acquires lock → credit estimate/check → create `AutomationRun` → stage methods query planner/writer models, call AI functions via `AIEngine`, respect batch sizes/delays, and update run state/logs → credits are tallied from `AITaskLog` differences.
|
|
||||||
- Billing: operations call `CreditService.check_*`/`deduct_*` before AI or content operations; invoices are created through `InvoiceService` and payments processed via `PaymentService`.
|
|
||||||
- Integration: API endpoints invoke `IntegrationService` to persist integrations, retrieve lists, and run connection tests; sync services handle subsequent data movement.
|
|
||||||
- Other domains: planner/content/publishing/linking/optimization services orchestrate their models and, where applicable, AI or external adapters; see domain docs for invocation points.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Automation stages consume planner (Keywords/Clusters/ContentIdeas) and writer (Tasks/Content/Images) data and rely on credit usage logs from AI tasks.
|
|
||||||
- Billing services are used by automation/AI flows to enforce credit availability and record deductions.
|
|
||||||
- Integration services connect site data and publishing/sync flows to external platforms; publishing services depend on integration metadata when targeting destinations.
|
|
||||||
- Planner/content services feed data used by publishing and optimization tasks.
|
|
||||||
|
|
||||||
## State Transitions (if applicable)
|
|
||||||
- Automation runs move through stages, can pause/cancel, and record partial progress with stage result JSON.
|
|
||||||
- Credit balances mutate through add/deduct operations; transactions/usage logs capture each change.
|
|
||||||
- Invoices progress through draft/pending/paid/void and payments through their status lifecycle.
|
|
||||||
- Integrations toggle active/sync flags and update sync status/errors.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Automation: checks for concurrent runs; validates minimum keywords; pauses/cancels mid-stage; writes stage error messages; releases locks on failures.
|
|
||||||
- Billing: raises when credit cost unknown or balance insufficient; wraps changes in atomic transactions.
|
|
||||||
- Integration: platform test errors are logged and returned with `success=false`; unsupported platforms raise `NotImplementedError`.
|
|
||||||
- Invoice service prevents voiding paid invoices and returns placeholder PDF until implemented.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Services operate on tenant-scoped models; constructors typically receive account/site or derive them from models. Integration creation sets account from site; credit operations mutate `Account.credits`.
|
|
||||||
- Privileged role bypass applies at the viewset layer; persisted records maintain account/site ownership.
|
|
||||||
|
|
||||||
## Billing Rules (if applicable)
|
|
||||||
- Costs resolved via `CreditCostConfig` (preferred) or constants; units can be per request, words (100/200), item, or image.
|
|
||||||
- Deduct operations both adjust `Account.credits` and log to `CreditTransaction` and `CreditUsageLog`.
|
|
||||||
- Invoice creation links to subscriptions or credit packages and uses account billing email/plan pricing.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers (if applicable)
|
|
||||||
- Automation uses Celery for image generation and may be triggered by scheduled runs (frequency/time in `AutomationConfig`).
|
|
||||||
- Other long-running tasks (publishing, optimization, sync) are handled via their Celery tasks/adapters in respective service modules.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Automation enforces exclusivity per site and accounts for credit sufficiency before starting.
|
|
||||||
- Logging (AutomationLogger) produces per-run artifacts and structured traces for observability.
|
|
||||||
- Credit handling is centralized to keep ledger and usage logs consistent and atomic.
|
|
||||||
- Integration services abstract platform handling and allow per-platform test logic.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Reuse `AutomationService` for running/continuing automation; respect locks and stage APIs.
|
|
||||||
- Use `CreditService` before AI/content-heavy operations to check/deduct/add credits and to log usage.
|
|
||||||
- Create invoices via `InvoiceService` helpers rather than constructing manually; update invoice/payment status through service methods.
|
|
||||||
- Manage integrations through `IntegrationService` (create/update/test/list) and extend platform-specific tests as needed.
|
|
||||||
- For domain-specific flows (planner/content/publishing/linking/optimization), place orchestration in the existing service modules and keep tenant context explicit.
|
|
||||||
@@ -1,62 +0,0 @@
|
|||||||
# Automation Logging
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Explain how automation runs are logged, where artifacts are written, and how logs are retrieved via API.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Logger implementation: `backend/igny8_core/business/automation/services/automation_logger.py`
|
|
||||||
- Service usage: `backend/igny8_core/business/automation/services/automation_service.py`
|
|
||||||
- API exposure: `backend/igny8_core/business/automation/views.py` (`logs` action)
|
|
||||||
- Run metadata: `backend/igny8_core/business/automation/models.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Create per-run directories and log files for automation runs (primary and optional shared mirror).
|
|
||||||
- Record stage-level start/progress/complete events and structured traces.
|
|
||||||
- Provide tail retrieval of logs through an API endpoint.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- `AutomationLogger`:
|
|
||||||
- On `start_run`: generates `run_id` (`run_YYYYMMDD_HHMMSS_<trigger>`), creates run directory under `base_log_dir` (default `/data/app/logs/automation`), and optionally mirrors to `shared_log_dir` when configured. Initializes `automation_run.log` with run metadata and writes a JSONL trace entry (`run_trace.jsonl`).
|
|
||||||
- `log_stage_start`: appends to main log and writes a stage-specific log file `stage_<n>.log`; mirrors to shared dir if configured; writes a trace event (`stage_start` with pending count).
|
|
||||||
- `log_stage_progress`: appends progress lines to main and stage logs (and mirrored copies), and writes a `stage_progress` trace event.
|
|
||||||
- `log_stage_complete`: writes completion info with processed count, time elapsed, credits used to main/stage logs; emits trace (`stage_complete`).
|
|
||||||
- `log_stage_error`: writes error lines to logs and emits an error trace.
|
|
||||||
- `append_trace`: helper used by the service for per-item events; appends JSONL entries.
|
|
||||||
- `get_activity_log`: reads and returns the tail of `automation_run.log` (used by API).
|
|
||||||
- `AutomationService` uses the logger at every stage boundary and significant progress point; also logs warnings/errors and mirrors events via the logger’s trace helpers.
|
|
||||||
- Stage result data is stored in `AutomationRun.stage_n_result` JSON fields alongside logs, providing structured state in the DB.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `AutomationRun`: stores per-stage result JSON and run status used to correlate with log files.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Run start → `start_run` creates directories/files → each stage emits start/progress/complete/error logs and traces → API `logs` endpoint reads `automation_run.log` tail for a given `run_id`.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Logging uses the automation run/account/site context; no external modules write to these files. AI stages provide progress data; billing is inferred from AI task logs but not logged here.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Log files accumulate throughout a run; traces document run lifecycle events. Completion or failure is also reflected in `AutomationRun` status/results.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Logger operations are best-effort; exceptions during file/trace writes are swallowed to avoid breaking the run.
|
|
||||||
- API `logs` returns 404 if the run does not exist; otherwise returns the requested number of lines.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- Run directories are keyed by account/site/run; API access is constrained by site ownership through the viewset query for `AutomationRun`.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- None directly; credit use is recorded in stage result JSON and inferred from AI task logs, not in the file logs.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- Logging occurs inside Celery-executed stages; no separate scheduled logging tasks.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- File-based logs per run make multi-stage processes auditable and debuggable without DB bloat.
|
|
||||||
- Optional shared log dir supports centralized aggregation when multiple containers run workers.
|
|
||||||
- JSONL traces allow machine parsing for UI status feeds while text logs support human debugging.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- When adding stage steps, emit progress via `AutomationLogger` to keep observability consistent.
|
|
||||||
- Keep log writes best-effort; do not raise from logger helpers.
|
|
||||||
- Ensure any new API log endpoints read from the same per-run directory structure and remain site/account scoped.
|
|
||||||
@@ -1,91 +0,0 @@
|
|||||||
# Automation Module Reference
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Document how the automation module orchestrates multi-stage AI pipelines, exposes API endpoints, enforces tenancy/credits, and manages runs, configs, and logging.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Models: `backend/igny8_core/business/automation/models.py`
|
|
||||||
- Services: `backend/igny8_core/business/automation/services/automation_service.py`, `automation_logger.py`
|
|
||||||
- Tasks (Celery): `backend/igny8_core/business/automation/tasks.py`
|
|
||||||
- API views and routing: `backend/igny8_core/business/automation/views.py`, `urls.py`
|
|
||||||
- Supporting AI functions: `backend/igny8_core/ai/functions/auto_cluster.py`, `generate_ideas.py`, `generate_content.py`, `generate_image_prompts.py`, image queue in `backend/igny8_core/ai/tasks.py`
|
|
||||||
- Tenancy/auth context: `backend/igny8_core/auth/middleware.py`, `backend/igny8_core/api/base.py`
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Maintain per-site automation configs (batch sizes, delays, schedule, enable flag) and track run state with detailed per-stage results.
|
|
||||||
- Provide APIs to configure, trigger, pause/resume/cancel, inspect, and log automation runs.
|
|
||||||
- Execute seven sequential stages that transform planner/writer data via AI and local operations, with credit checks and pause/cancel handling.
|
|
||||||
- Enforce tenant/site scoping on all automation resources and API operations.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
- `AutomationConfig` stores enablement, frequency, scheduled time, batch sizes for stages 1–6, and within/between-stage delays. Config is created lazily per site.
|
|
||||||
- `AutomationRun` captures run metadata: trigger type (manual/scheduled), status (`running/paused/cancelled/completed/failed`), current stage, pause/cancel timestamps, per-stage JSON results, total credits used, and error message.
|
|
||||||
- `AutomationService` orchestrates the pipeline:
|
|
||||||
- Locks per site via cache (`automation_lock_{site.id}`) to prevent concurrent runs.
|
|
||||||
- Estimates credits before start and requires a 20% buffer over the estimate against `Account.credits`.
|
|
||||||
- Creates `AutomationRun` with generated `run_id` and logs start via `AutomationLogger`.
|
|
||||||
- Executes stages in order; each stage logs start/progress/complete, applies within/between-stage delays from config, and writes stage result JSON (counts, credits, timestamps, partial flags).
|
|
||||||
- Pause/cancel checks occur inside loops; state is persisted so resumed runs continue from the recorded stage.
|
|
||||||
- Stage credit usage is derived from AI task logs difference before/after the stage.
|
|
||||||
- API layer (`AutomationViewSet`):
|
|
||||||
- `config`/`update_config` read/write `AutomationConfig` for a given `site_id` (scoped to the user’s account).
|
|
||||||
- `run_now` triggers `AutomationService.start_automation` and enqueues Celery `run_automation_task`.
|
|
||||||
- `current_run`, `history`, `logs`, `current_processing`, `estimate`, `pipeline_overview` expose run status, history, logs, credit estimates, and per-stage pending counts.
|
|
||||||
- `pause`, `resume`, `cancel` endpoints update run status and enqueue resume tasks when needed.
|
|
||||||
- Celery tasks:
|
|
||||||
- `check_scheduled_automations` scans enabled configs hourly and triggers runs when frequency/time matches and no recent run exists.
|
|
||||||
- `run_automation_task` performs full pipeline execution.
|
|
||||||
- `resume_automation_task`/`continue_automation_task` continue a paused run from its recorded stage.
|
|
||||||
|
|
||||||
## Data Structures / Models Involved (no code)
|
|
||||||
- `AutomationConfig`, `AutomationRun` (automation state).
|
|
||||||
- Planner models: `Keywords`, `Clusters`, `ContentIdeas`.
|
|
||||||
- Writer models: `Tasks`, `Content`, `Images`.
|
|
||||||
- AI task log (`AITaskLog`) for credit usage measurement.
|
|
||||||
- Tenancy entities: `Account`, `Site` (scoping every query).
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- API call → DRF auth → tenant/site resolved → viewset method → `AutomationService` operations → Celery task (for long-running execution).
|
|
||||||
- Pipeline stages run in-process inside Celery workers, reading planner/writer data, invoking AI functions, updating models, logging progress, and writing stage results to `AutomationRun`.
|
|
||||||
- Completion (or failure) updates run status and releases the site lock.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Planner/writer models supply inputs and receive outputs (clusters, ideas, tasks, content, images).
|
|
||||||
- AI engine executes clustering, idea generation, content generation, and image prompt generation; image rendering uses the AI image queue.
|
|
||||||
- Billing credits are checked against `Account.credits`; credit usage is inferred from AI task logs (deduction logic handled in billing services when those AI calls occur).
|
|
||||||
- Integration/publishing modules consume content/images produced downstream (outside automation).
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- Run status moves through `running` → (`paused`/`cancelled`/`failed`/`completed`); `current_stage` increments after each stage finishes; partial flags and timestamps mark mid-stage exits.
|
|
||||||
- Config changes take effect on the next run; pause/resume toggles update run timestamps.
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Start blocks if a run is already active for the site or cache lock is held.
|
|
||||||
- Stage loops log and continue on per-batch/item errors; pause/cancel results are persisted mid-stage.
|
|
||||||
- Failures in Celery run mark `AutomationRun` as failed, store error message, timestamp completion, and release the lock.
|
|
||||||
- API endpoints return 400 for missing params or invalid state transitions, 404 for unknown runs, 500 on unexpected errors.
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All automation queries filter by `site` tied to the authenticated user’s `account`; config/run creation sets `account` and `site` explicitly.
|
|
||||||
- API endpoints fetch `Site` with `account=request.user.account`; automation locks are per site.
|
|
||||||
- No cross-tenant access; privileged role bypass is handled by DRF auth/permissions upstream.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- Start requires `Account.credits` ≥ 1.2× estimated credits; otherwise a 400 is returned.
|
|
||||||
- Credits actually deducted by AI tasks are reflected via AI task logs and billing services (outside this module); automation aggregates usage per stage in `AutomationRun`.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- Hourly `check_scheduled_automations` respects config frequency/time and last run; skips if a run is already active.
|
|
||||||
- Pipeline execution and resume steps run inside Celery tasks; within-stage sleeps apply delays from config.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Single-run-per-site enforced via cache lock to prevent overlapping credit use or data contention.
|
|
||||||
- Pause/resume/cancel is cooperative, checked inside stage loops, with partial results persisted.
|
|
||||||
- Stage-by-stage logging and result JSON make pipeline progress observable and resumable.
|
|
||||||
- Configurable batch sizes and delays balance throughput and API/credit usage.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- Use `AutomationService.start_automation` for new runs; never bypass the cache lock or credit check.
|
|
||||||
- When extending stages, preserve pause/cancel checks, result recording, and credit delta calculation.
|
|
||||||
- Add new API actions through `AutomationViewSet` if they manipulate automation state; keep site/account scoping.
|
|
||||||
- For new schedulers, reuse the lock pattern and `AutomationConfig` fields, and update `next_run_at` appropriately.
|
|
||||||
@@ -1,102 +0,0 @@
|
|||||||
# Automation Pipeline Stages
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
Detail the seven pipeline stages executed by `AutomationService`, including inputs, queries, validations, delays, credit handling, and state recording.
|
|
||||||
|
|
||||||
## Code Locations (exact paths)
|
|
||||||
- Orchestration: `backend/igny8_core/business/automation/services/automation_service.py`
|
|
||||||
- Models: `backend/igny8_core/business/automation/models.py`
|
|
||||||
- AI functions: `backend/igny8_core/ai/functions/auto_cluster.py`, `generate_ideas.py`, `generate_content.py`, `generate_image_prompts.py`
|
|
||||||
- Image queue: `backend/igny8_core/ai/tasks.py` (`process_image_generation_queue`)
|
|
||||||
- Stage entrypoints: `backend/igny8_core/business/automation/tasks.py` (Celery `run_automation_task`, `resume_automation_task`)
|
|
||||||
|
|
||||||
## High-Level Responsibilities
|
|
||||||
- Execute a fixed seven-stage sequence that moves data from planner keywords through content with images and into manual review readiness.
|
|
||||||
- Enforce batch sizes/delays from `AutomationConfig`, support pause/cancel, and write per-stage results into `AutomationRun`.
|
|
||||||
- Track credit deltas per stage using AI task log counts.
|
|
||||||
|
|
||||||
## Detailed Behavior
|
|
||||||
Across all stages:
|
|
||||||
- Each stage logs start/progress/complete via `AutomationLogger`, respects `within_stage_delay` between batches/items, and `between_stage_delay` between stages.
|
|
||||||
- Pause/cancel is checked inside loops; on pause/cancel, the stage records partial counts, credits, elapsed time, and reason, then exits.
|
|
||||||
- Credits used per stage are computed from `AITaskLog` count delta relative to stage start.
|
|
||||||
|
|
||||||
### Stage 1: Keywords → Clusters (AI)
|
|
||||||
- Input query: `Keywords` where `site=current`, `status='new'`, `cluster__isnull=True`, `disabled=False`.
|
|
||||||
- Validation: `validate_minimum_keywords` requires at least 5 keywords; if not valid, stage is skipped with result noting skip reason and `current_stage` advances to 2.
|
|
||||||
- Processing: Batch size = `stage_1_batch_size` (capped to total). For each batch, calls `AIEngine.execute(AutoClusterFunction, payload={'ids': batch})`; waits on task ID; logs per-batch progress. Errors are logged and skipped; pipeline continues.
|
|
||||||
- Result: counts keywords processed, clusters created since run start, batches, credits used, time elapsed; sets `current_stage=2`.
|
|
||||||
|
|
||||||
### Stage 2: Clusters → Ideas (AI)
|
|
||||||
- Pre-check: warns if any `Keywords` still pending from Stage 1.
|
|
||||||
- Input query: `Clusters` where `site=current`, `status='new'`, `disabled=False`.
|
|
||||||
- Processing: Iterates clusters one-by-one; for each, calls `AIEngine.execute(GenerateIdeasFunction, payload={'cluster_id': cluster.id})`; waits on task ID; logs progress. Errors are logged and skipped.
|
|
||||||
- Result: counts clusters processed, ideas created since run start, credits used, time elapsed; sets `current_stage=3`.
|
|
||||||
|
|
||||||
### Stage 3: Ideas → Tasks (Local)
|
|
||||||
- Pre-check: warns if clusters remain without ideas.
|
|
||||||
- Input query: `ContentIdeas` where `site=current`, `status='new'`.
|
|
||||||
- Processing: Batched by `stage_3_batch_size`. For each idea, builds keyword string (M2M keywords or `target_keywords`) and creates `Tasks` with queued status, copying account/site/sector, cluster, content type/structure, and description. Idea status set to `queued`.
|
|
||||||
- Result: ideas processed, tasks created, batches, time elapsed (credits 0 because local); sets `current_stage=4`.
|
|
||||||
|
|
||||||
### Stage 4: Tasks → Content (AI)
|
|
||||||
- Pre-check: warns if `ContentIdeas` remain `new`.
|
|
||||||
- Input query: `Tasks` where `site=current`, `status='queued'`.
|
|
||||||
- Processing: Batched by `stage_4_batch_size`. Uses `GenerateContentFunction` via `AIEngine` per batch (payload contains task IDs). Waits on task IDs, logs progress, continues on errors. Tracks total words by summing generated content word_count.
|
|
||||||
- Result: tasks processed, content created count, total_words, credits used, time elapsed; sets `current_stage=5`.
|
|
||||||
|
|
||||||
### Stage 5: Content → Image Prompts (AI)
|
|
||||||
- Input query: `Content` where `site=current`, `status='draft'`, with zero images (annotated count=0).
|
|
||||||
- Processing: Batched by `stage_5_batch_size`. For each batch, calls `GenerateImagePromptsFunction` via `AIEngine` (payload content IDs). Waits on task IDs, logs progress; continues on errors.
|
|
||||||
- Result: content processed, prompts created (from AI task logs), credits used, time elapsed; sets `current_stage=6`.
|
|
||||||
|
|
||||||
### Stage 6: Image Prompts → Images (AI image queue)
|
|
||||||
- Input query: `Images` where `site=current`, `status='pending'`.
|
|
||||||
- Processing: Iterates pending images; for each, enqueues `process_image_generation_queue.delay(image_ids=[id], account_id, content_id)` when Celery is available, or calls directly in sync fallback. Waits on task IDs with continue-on-error to avoid blocking the stage. Logs progress per image; applies within-stage delay between images.
|
|
||||||
- Result: images processed, images generated (status `generated` since run start), content moved to `review`, credits used, time elapsed; sets `current_stage=7`.
|
|
||||||
|
|
||||||
### Stage 7: Manual Review Gate (Count-only)
|
|
||||||
- Input query: `Content` where `site=current`, `status='review'`.
|
|
||||||
- Processing: Counts review-ready content, logs IDs (truncated), marks run `status='completed'`, sets `completed_at`, and releases the site lock.
|
|
||||||
- Result: ready_for_review count and content IDs stored in `stage_7_result`.
|
|
||||||
|
|
||||||
## Execution Flow
|
|
||||||
- Celery task `run_automation_task` instantiates `AutomationService.from_run_id` and calls stages 1→7 sequentially.
|
|
||||||
- Stage transitions update `AutomationRun.current_stage`; between-stage delays applied via `between_stage_delay`.
|
|
||||||
- Resume path (`resume_automation_task`) starts from the recorded `current_stage` and continues through remaining stages.
|
|
||||||
|
|
||||||
## Cross-Module Interactions
|
|
||||||
- Planner: Stage 1/2 use `Keywords`/`Clusters`; Stage 3 converts `ContentIdeas` into `Tasks`.
|
|
||||||
- Writer: Stages 4–6 create `Content` and `Images` and move content toward review.
|
|
||||||
- AI engine and functions are invoked in Stages 1, 2, 4, 5; Stage 6 uses the AI image queue.
|
|
||||||
- Billing: Credits are consumed by AI calls; automation records deltas per stage from AI task logs.
|
|
||||||
|
|
||||||
## State Transitions
|
|
||||||
- `AutomationRun.status` moves to `completed` at Stage 7; can be set to `failed` on exceptions or `cancelled` via API; `paused` can be set mid-run and resumed.
|
|
||||||
- `current_stage` increments after each successful stage; partial stage results include a `partial` flag and stop reason.
|
|
||||||
- Domain models change status along the pipeline (`Keywords` → clusters, `Clusters` → ideas, `ContentIdeas` → queued/tasks, `Tasks` → completed/content, `Content` → draft/review, `Images` → generated).
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
- Each stage logs errors and continues to next batch/item; pause/cancel checks short-circuit with partial results saved.
|
|
||||||
- Task wait helper tolerates Celery backend errors; can continue on error when flagged.
|
|
||||||
- Stage start may be skipped with explicit skip reason (e.g., insufficient keywords).
|
|
||||||
|
|
||||||
## Tenancy Rules
|
|
||||||
- All queries filter by `site` (and implicit account via tenancy bases); account/site set on created `Tasks` and inherited on `Images` and other records through model save hooks.
|
|
||||||
- Locks and runs are per site; API scoping requires the authenticated user’s account to own the site.
|
|
||||||
|
|
||||||
## Billing Rules
|
|
||||||
- Start requires sufficient credits (1.2× estimate). Credits used are inferred from AI task log counts per stage; actual deductions occur in AI/billing services invoked by the AI functions.
|
|
||||||
|
|
||||||
## Background Tasks / Schedulers
|
|
||||||
- Entire stage chain runs inside Celery workers; within-stage sleeps respect config delays; between-stage sleeps applied after each stage.
|
|
||||||
|
|
||||||
## Key Design Considerations
|
|
||||||
- Idempotent, resume-capable progression with partial state persisted in `AutomationRun`.
|
|
||||||
- Configurable batch sizes/delays mitigate rate limits and manage credit burn.
|
|
||||||
- Continue-on-error semantics prevent single failures from stopping the pipeline while still recording issues.
|
|
||||||
|
|
||||||
## How Developers Should Work With This Module
|
|
||||||
- When modifying stages, keep pause/cancel checks, stage result recording, and credit delta calculation.
|
|
||||||
- Add new AI stages by wiring through `AIEngine.execute` and the task wait helper; ensure queries are site-scoped and statuses updated.
|
|
||||||
- For new items types, add pending queries and status transitions consistent with existing patterns.
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user