new docs
This commit is contained in:
alorig
749
docs/01-ARCHITECTURE.md
Normal file
749
docs/01-ARCHITECTURE.md
Normal file
@@ -0,0 +1,749 @@
|
||||
# IGNY8 System Architecture
|
||||
|
||||
**Version:** 1.0
|
||||
**Last Updated:** 2025-01-XX
|
||||
**Purpose:** Complete system architecture documentation covering design patterns, principles, tech stack, and structural organization.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [System Overview](#system-overview)
|
||||
2. [Tech Stack](#tech-stack)
|
||||
3. [Core Architecture Principles](#core-architecture-principles)
|
||||
4. [Project Structure](#project-structure)
|
||||
5. [Key Architectural Patterns](#key-architectural-patterns)
|
||||
6. [Multi-Tenancy Architecture](#multi-tenancy-architecture)
|
||||
7. [Module Organization](#module-organization)
|
||||
8. [API Architecture](#api-architecture)
|
||||
9. [Frontend Architecture](#frontend-architecture)
|
||||
10. [Backend Architecture](#backend-architecture)
|
||||
11. [Database Architecture](#database-architecture)
|
||||
12. [Security Architecture](#security-architecture)
|
||||
13. [Deployment Architecture](#deployment-architecture)
|
||||
|
||||
---
|
||||
|
||||
## System Overview
|
||||
|
||||
**IGNY8** is a full-stack SaaS platform for SEO keyword management and AI-driven content generation. The system migrated from a WordPress plugin architecture to a modern Django + React architecture, providing a scalable, multi-account platform for content planning and generation.
|
||||
|
||||
### Core Capabilities
|
||||
|
||||
- **Multi-Account SaaS Platform**: Complete account isolation with site/sector hierarchy
|
||||
- **SEO Keyword Management**: Import, organize, and cluster keywords
|
||||
- **AI-Powered Content Planning**: Automated keyword clustering and content idea generation
|
||||
- **AI Content Generation**: Automated blog post and article generation
|
||||
- **Image Generation**: AI-powered image generation for content
|
||||
- **WordPress Integration**: Publish content directly to WordPress sites
|
||||
- **Subscription Management**: Plan-based limits and billing integration
|
||||
|
||||
---
|
||||
|
||||
## Tech Stack
|
||||
|
||||
### Backend
|
||||
|
||||
- **Framework**: Django 5.2+ with Django REST Framework (DRF)
|
||||
- **Database**: PostgreSQL
|
||||
- **Task Queue**: Celery with Redis broker
|
||||
- **Authentication**: JWT (JSON Web Tokens) + Session-based auth
|
||||
- **API**: RESTful API with DRF ViewSets
|
||||
- **Caching**: Redis
|
||||
- **File Storage**: Local filesystem (configurable for S3)
|
||||
|
||||
### Frontend
|
||||
|
||||
- **Framework**: React 19 with TypeScript
|
||||
- **Build Tool**: Vite
|
||||
- **Styling**: Tailwind CSS
|
||||
- **State Management**: Zustand
|
||||
- **Routing**: React Router v6
|
||||
- **HTTP Client**: Fetch API with custom wrapper
|
||||
- **UI Components**: Custom component library
|
||||
|
||||
### Infrastructure
|
||||
|
||||
- **Containerization**: Docker & Docker Compose
|
||||
- **Reverse Proxy**: Caddy (HTTPS on port 443)
|
||||
- **Process Management**: Supervisor (for Celery workers)
|
||||
- **Monitoring**: Portainer (optional)
|
||||
|
||||
### Development Tools
|
||||
|
||||
- **Backend**: Python 3.11+, pip, virtualenv
|
||||
- **Frontend**: Node.js 18+, npm/yarn
|
||||
- **Version Control**: Git
|
||||
- **Code Quality**: ESLint, Prettier (frontend), Black, Flake8 (backend)
|
||||
|
||||
---
|
||||
|
||||
## Core Architecture Principles
|
||||
|
||||
### 1. Configuration-Driven Everything
|
||||
|
||||
**Principle**: Zero HTML/JSX duplication - All UI rendered from configuration.
|
||||
|
||||
- **Tables, filters, forms** all driven by config files
|
||||
- **Single source of truth** - Change config, UI updates everywhere
|
||||
- **Page-local config** for page-specific settings
|
||||
- **Shared snippets** for reusable column/filter/action definitions
|
||||
|
||||
**Implementation**:
|
||||
- Frontend: Config files in `/config/pages/` and `/config/snippets/`
|
||||
- Backend: DRF serializers and ViewSet actions
|
||||
|
||||
### 2. Multi-Tenancy Foundation
|
||||
|
||||
**Principle**: Complete account isolation with automatic filtering.
|
||||
|
||||
- All models inherit `AccountBaseModel` with automatic account isolation
|
||||
- All ViewSets inherit `AccountModelViewSet` with automatic account filtering
|
||||
- Middleware injects account context from JWT on every request
|
||||
- Site > Sector hierarchy for content organization
|
||||
|
||||
**Implementation**:
|
||||
- Base models: `AccountBaseModel`, `SiteSectorBaseModel`
|
||||
- Base ViewSets: `AccountModelViewSet`, `SiteSectorModelViewSet`
|
||||
- Middleware: `AccountContextMiddleware` sets `request.account`
|
||||
|
||||
### 3. Template System (4 Universal Templates)
|
||||
|
||||
**Principle**: Reusable templates for all page types.
|
||||
|
||||
- **DashboardTemplate**: Module home pages (KPIs, workflow steps, charts)
|
||||
- **TablePageTemplate**: CRUD table pages (Keywords, Clusters, Tasks, etc.)
|
||||
- **FormPageTemplate**: Settings/form pages (Settings, Integration, etc.)
|
||||
- **SystemPageTemplate**: System/admin pages (Logs, Status, Monitoring)
|
||||
|
||||
**Implementation**:
|
||||
- Frontend: `/templates/` directory with 4 template components
|
||||
- Config-driven: Templates accept config objects for customization
|
||||
|
||||
### 4. Unified AI Processor
|
||||
|
||||
**Principle**: Single interface for all AI operations.
|
||||
|
||||
- Single `AIProcessor` class handles all AI operations
|
||||
- Manual and automated workflows use same functions
|
||||
- Action-based routing: 'clustering', 'ideas', 'content_generation', 'image_generation'
|
||||
- Account-specific API keys and model configuration
|
||||
|
||||
**Implementation**:
|
||||
- Backend: `AIProcessor` class in `/utils/ai_processor.py`
|
||||
- Integration: Loads API keys from `IntegrationSettings` model
|
||||
|
||||
### 5. Module-Based Organization
|
||||
|
||||
**Principle**: Clear module boundaries with shared utilities.
|
||||
|
||||
- Each module = Django app (`/modules/{module}/`)
|
||||
- Clear module boundaries with shared utilities
|
||||
- Module router pattern for subpage routing
|
||||
- Consistent structure across modules
|
||||
|
||||
**Implementation**:
|
||||
- Backend: `/modules/planner/`, `/modules/writer/`, `/modules/system/`, `/modules/billing/`
|
||||
- Frontend: `/pages/Planner/`, `/pages/Writer/`, `/pages/Settings/`, etc.
|
||||
|
||||
---
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
igny8/
|
||||
├── backend/ # Django backend
|
||||
│ └── igny8_core/ # Django project
|
||||
│ ├── auth/ # Multi-tenancy, User, Account, Plan models
|
||||
│ │ ├── models.py # Account, User, Plan, Site, Sector, Industry models
|
||||
│ │ ├── views.py # Account, User, Site, Sector ViewSets
|
||||
│ │ ├── serializers.py # Account, User, Plan serializers
|
||||
│ │ └── urls.py # Auth module URLs
|
||||
│ ├── modules/ # Feature modules
|
||||
│ │ ├── planner/ # Keywords, Clusters, Ideas
|
||||
│ │ │ ├── models.py # Keywords, Clusters, ContentIdeas models
|
||||
│ │ │ ├── views.py # KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
|
||||
│ │ │ ├── tasks.py # Celery tasks for AI operations
|
||||
│ │ │ ├── serializers.py # Model serializers
|
||||
│ │ │ └── urls.py # Planner module URLs
|
||||
│ │ ├── writer/ # Tasks, Content, Images
|
||||
│ │ │ ├── models.py # Tasks, Content, Images models
|
||||
│ │ │ ├── views.py # TasksViewSet
|
||||
│ │ │ ├── tasks.py # Celery tasks for content/image generation
|
||||
│ │ │ └── urls.py # Writer module URLs
|
||||
│ │ ├── system/ # Settings, Prompts, Integration
|
||||
│ │ │ ├── models.py # AIPrompt, IntegrationSettings, AuthorProfile, Strategy
|
||||
│ │ │ ├── views.py # AIPromptViewSet, AuthorProfileViewSet
|
||||
│ │ │ ├── integration_views.py # IntegrationSettingsViewSet, task_progress
|
||||
│ │ │ ├── utils.py # Default prompts, prompt loading
|
||||
│ │ │ └── urls.py # System module URLs
|
||||
│ │ └── billing/ # Credits, Transactions, Usage
|
||||
│ │ ├── models.py # CreditTransaction, UsageLog models
|
||||
│ │ ├── views.py # Billing ViewSets
|
||||
│ │ └── services.py # CreditService
|
||||
│ ├── api/ # API base classes
|
||||
│ │ └── base.py # AccountModelViewSet, SiteSectorModelViewSet
|
||||
│ ├── utils/ # Shared utilities
|
||||
│ │ ├── ai_processor.py # Unified AI interface
|
||||
│ │ └── content_normalizer.py # Content processing utilities
|
||||
│ ├── middleware/ # Custom middleware
|
||||
│ │ ├── account.py # AccountContextMiddleware (sets request.account)
|
||||
│ │ └── resource_tracker.py # ResourceTrackerMiddleware (API metrics)
|
||||
│ ├── settings.py # Django settings
|
||||
│ ├── urls.py # Root URL configuration
|
||||
│ └── celery.py # Celery configuration
|
||||
│
|
||||
├── frontend/ # React frontend
|
||||
│ └── src/
|
||||
│ ├── pages/ # Page components
|
||||
│ │ ├── Planner/ # KeywordsPage, ClustersPage, IdeasPage, Dashboard
|
||||
│ │ ├── Writer/ # TasksPage, DraftsPage, PublishedPage, Dashboard
|
||||
│ │ ├── Settings/ # General, Integration, Status, ImportExport
|
||||
│ │ ├── Billing/ # Credits, Transactions, Usage
|
||||
│ │ └── AuthPages/ # SignIn, SignUp
|
||||
│ ├── templates/ # 4 master templates
|
||||
│ │ ├── DashboardTemplate.tsx
|
||||
│ │ ├── TablePageTemplate.tsx
|
||||
│ │ ├── FormPageTemplate.tsx
|
||||
│ │ └── SystemPageTemplate.tsx
|
||||
│ ├── components/ # UI components
|
||||
│ │ ├── layout/ # AppLayout, Sidebar, Header, Breadcrumbs
|
||||
│ │ ├── table/ # DataTable, Filters, Actions, Pagination
|
||||
│ │ ├── ui/ # Button, Card, Modal, Toast, etc.
|
||||
│ │ └── auth/ # ProtectedRoute, Auth components
|
||||
│ ├── config/ # Configuration files
|
||||
│ │ ├── pages/ # Page-specific configs
|
||||
│ │ │ └── keywords.config.tsx
|
||||
│ │ ├── snippets/ # Shared column/filter/action definitions
|
||||
│ │ │ ├── columns.snippets.ts
|
||||
│ │ │ ├── filters.snippets.ts
|
||||
│ │ │ └── actions.snippets.ts
|
||||
│ │ └── routes.config.ts # Route configuration
|
||||
│ ├── store/ # Zustand stores
|
||||
│ │ ├── authStore.ts # Authentication state
|
||||
│ │ ├── plannerStore.ts # Planner module state
|
||||
│ │ ├── siteStore.ts # Site selection state
|
||||
│ │ └── aiRequestLogsStore.ts # AI request/response logs
|
||||
│ ├── services/ # API clients
|
||||
│ │ └── api.ts # fetchAPI, API functions
|
||||
│ ├── hooks/ # Custom React hooks
|
||||
│ │ ├── useProgressModal.ts # Progress modal for long-running tasks
|
||||
│ │ └── useAuth.ts # Authentication hook
|
||||
│ ├── layout/ # Layout components
|
||||
│ │ └── AppLayout.tsx # Main app layout wrapper
|
||||
│ ├── App.tsx # Root component with routing
|
||||
│ └── main.tsx # Entry point
|
||||
│
|
||||
└── docs/ # Documentation
|
||||
└── ActiveDocs/ # Active documentation
|
||||
├── 01-ARCHITECTURE.md # This file
|
||||
├── 02-FRONTEND.md # Frontend documentation
|
||||
├── 03-BACKEND.md # Backend documentation
|
||||
├── 04-AI-FUNCTIONS.md # AI functions documentation
|
||||
└── 05-ACCOUNT-USER-PLAN.md # Account/User/Plan documentation
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Architectural Patterns
|
||||
|
||||
### Configuration System (Page-Local Config + Shared Snippets)
|
||||
|
||||
**Rule**: Config = Page-Local, Snippets = Shared
|
||||
|
||||
**Structure**:
|
||||
```
|
||||
/pages/Planner/KeywordsPage.tsx
|
||||
├── Imports snippets from /config/snippets/
|
||||
├── Defines page-local tableConfig, filterConfig, actionsConfig
|
||||
└── Passes config to TablePageTemplate
|
||||
|
||||
/config/snippets/
|
||||
├── columns.snippets.ts # statusColumn, titleColumn, etc.
|
||||
├── filters.snippets.ts # statusFilter, dateRangeFilter, etc.
|
||||
├── actions.snippets.ts # commonActions, bulkActions, etc.
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- Reusable components across pages
|
||||
- Page-specific customization
|
||||
- Single source of truth for shared definitions
|
||||
|
||||
### Base ViewSet Pattern
|
||||
|
||||
**Backend Pattern**: All ViewSets inherit from base classes for consistent behavior.
|
||||
|
||||
**Base Classes**:
|
||||
- `AccountModelViewSet`: Automatic account filtering
|
||||
- `SiteSectorModelViewSet`: Account + site/sector filtering + access control
|
||||
|
||||
**Benefits**:
|
||||
- Consistent access control
|
||||
- Automatic account isolation
|
||||
- Reduced code duplication
|
||||
|
||||
### Celery Task Pattern
|
||||
|
||||
**Pattern**: Long-running operations use Celery tasks with progress tracking.
|
||||
|
||||
**Structure**:
|
||||
1. API endpoint queues Celery task
|
||||
2. Task updates progress via `update_state()`
|
||||
3. Frontend polls `task_progress` endpoint
|
||||
4. Progress displayed in modal
|
||||
|
||||
**Benefits**:
|
||||
- Non-blocking API responses
|
||||
- Real-time progress updates
|
||||
- Scalable background processing
|
||||
|
||||
### AI Function Pattern
|
||||
|
||||
**Pattern**: All AI functions follow consistent structure.
|
||||
|
||||
**Structure**:
|
||||
1. API Endpoint: Validates input, queues Celery task
|
||||
2. Celery Task: Wraps core function with progress tracking
|
||||
3. Core Function: Business logic, calls AIProcessor
|
||||
4. AIProcessor: Makes API calls, returns structured data
|
||||
5. Core Function: Saves results to database
|
||||
|
||||
**Benefits**:
|
||||
- Consistent error handling
|
||||
- Progress tracking
|
||||
- Reusable AI interface
|
||||
|
||||
---
|
||||
|
||||
## Multi-Tenancy Architecture
|
||||
|
||||
### Account Isolation
|
||||
|
||||
**Principle**: All data is isolated by account.
|
||||
|
||||
**Implementation**:
|
||||
- All models inherit `AccountBaseModel` (has `account` ForeignKey)
|
||||
- All ViewSets inherit `AccountModelViewSet` (filters by `request.account`)
|
||||
- Middleware sets `request.account` from JWT token
|
||||
|
||||
**Access Control**:
|
||||
- Admin/Developer users: Bypass account filtering (see all accounts)
|
||||
- System account users: Bypass account filtering (see all accounts)
|
||||
- Regular users: Only see data from their account
|
||||
|
||||
### Site/Sector Hierarchy
|
||||
|
||||
**Structure**:
|
||||
```
|
||||
Account (1) ──< (N) Site
|
||||
Site (1) ──< (1-5) Sector
|
||||
Sector (1) ──< (N) Keywords, Clusters, ContentIdeas, Tasks
|
||||
```
|
||||
|
||||
**Implementation**:
|
||||
- Models inherit `SiteSectorBaseModel` (has `site` and `sector` ForeignKeys)
|
||||
- ViewSets inherit `SiteSectorModelViewSet` (filters by accessible sites)
|
||||
- User access control via `User.get_accessible_sites()`
|
||||
|
||||
**Site Access Control**:
|
||||
- System account users: All active sites
|
||||
- Developers: All active sites
|
||||
- Owners/Admins: All sites in their account
|
||||
- Editors/Viewers: Only sites granted via `SiteUserAccess`
|
||||
|
||||
---
|
||||
|
||||
## Module Organization
|
||||
|
||||
### Planner Module
|
||||
|
||||
**Purpose**: Keyword management and content planning.
|
||||
|
||||
**Models**:
|
||||
- `Keywords`: Individual keywords with volume, difficulty, intent
|
||||
- `Clusters`: Keyword clusters (groups of related keywords)
|
||||
- `ContentIdeas`: Content ideas generated from clusters
|
||||
|
||||
**ViewSets**:
|
||||
- `KeywordViewSet`: CRUD + `auto_cluster` action
|
||||
- `ClusterViewSet`: CRUD + `auto_generate_ideas` action
|
||||
- `ContentIdeasViewSet`: CRUD operations
|
||||
|
||||
**Tasks**:
|
||||
- `auto_cluster_keywords_task`: AI-powered keyword clustering
|
||||
- `auto_generate_ideas_task`: AI-powered content idea generation
|
||||
|
||||
### Writer Module
|
||||
|
||||
**Purpose**: Content generation and management.
|
||||
|
||||
**Models**:
|
||||
- `Tasks`: Content generation tasks
|
||||
- `Content`: Generated content (HTML)
|
||||
- `Images`: Generated images for tasks
|
||||
|
||||
**ViewSets**:
|
||||
- `TasksViewSet`: CRUD + `auto_generate_content`, `auto_generate_images` actions
|
||||
|
||||
**Tasks**:
|
||||
- `auto_generate_content_task`: AI-powered content generation
|
||||
- `auto_generate_images_task`: AI-powered image generation
|
||||
|
||||
### System Module
|
||||
|
||||
**Purpose**: System settings, prompts, and integrations.
|
||||
|
||||
**Models**:
|
||||
- `AIPrompt`: AI prompt templates (clustering, ideas, content, images)
|
||||
- `IntegrationSettings`: API keys and configuration (OpenAI, Runware, etc.)
|
||||
- `AuthorProfile`: Writing style profiles
|
||||
- `Strategy`: Content strategies per sector
|
||||
|
||||
**ViewSets**:
|
||||
- `AIPromptViewSet`: CRUD for prompts
|
||||
- `IntegrationSettingsViewSet`: CRUD + `test_openai`, `test_runware`, `generate_image`, `task_progress` actions
|
||||
- `AuthorProfileViewSet`: CRUD for author profiles
|
||||
- `StrategyViewSet`: CRUD for strategies
|
||||
|
||||
### Billing Module
|
||||
|
||||
**Purpose**: Credits, transactions, and usage tracking.
|
||||
|
||||
**Models**:
|
||||
- `CreditTransaction`: Credit purchase/usage transactions
|
||||
- `UsageLog`: Daily/monthly usage tracking
|
||||
|
||||
**ViewSets**:
|
||||
- `CreditTransactionViewSet`: CRUD for transactions
|
||||
- `UsageLogViewSet`: Read-only usage logs
|
||||
|
||||
**Services**:
|
||||
- `CreditService`: Credit calculation and deduction logic
|
||||
|
||||
---
|
||||
|
||||
## API Architecture
|
||||
|
||||
### RESTful API Design
|
||||
|
||||
**Base URL**: `/api/v1/`
|
||||
|
||||
**Endpoint Structure**:
|
||||
- `/api/v1/planner/keywords/` - Keywords CRUD
|
||||
- `/api/v1/planner/keywords/auto_cluster/` - Auto-cluster action
|
||||
- `/api/v1/planner/clusters/` - Clusters CRUD
|
||||
- `/api/v1/planner/clusters/auto_generate_ideas/` - Auto-generate ideas action
|
||||
- `/api/v1/writer/tasks/` - Tasks CRUD
|
||||
- `/api/v1/writer/tasks/auto_generate_content/` - Auto-generate content action
|
||||
- `/api/v1/system/settings/task_progress/{task_id}/` - Task progress polling
|
||||
|
||||
### Authentication
|
||||
|
||||
**Methods**:
|
||||
- JWT (JSON Web Tokens) - Primary method
|
||||
- Session-based auth - Fallback for admin
|
||||
|
||||
**Flow**:
|
||||
1. User signs in → Backend returns JWT token
|
||||
2. Frontend stores token in localStorage
|
||||
3. Frontend includes token in `Authorization: Bearer {token}` header
|
||||
4. Backend middleware validates token and sets `request.user` and `request.account`
|
||||
|
||||
### Response Format
|
||||
|
||||
**Success Response**:
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": { ... },
|
||||
"message": "Optional message"
|
||||
}
|
||||
```
|
||||
|
||||
**Error Response**:
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"message": "Error message",
|
||||
"errors": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
### Pagination
|
||||
|
||||
**Format**: Page-based pagination
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"count": 100,
|
||||
"next": "http://api.example.com/api/v1/resource/?page=2",
|
||||
"previous": null,
|
||||
"results": [ ... ]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Frontend Architecture
|
||||
|
||||
### Component Hierarchy
|
||||
|
||||
```
|
||||
App
|
||||
└── AppLayout
|
||||
├── Sidebar (navigation)
|
||||
├── Header (user menu, notifications)
|
||||
└── Main Content
|
||||
└── Page Component
|
||||
└── Template (DashboardTemplate, TablePageTemplate, etc.)
|
||||
└── Components (DataTable, Filters, etc.)
|
||||
```
|
||||
|
||||
### State Management
|
||||
|
||||
**Zustand Stores**:
|
||||
- `authStore`: Authentication state (user, token, account)
|
||||
- `plannerStore`: Planner module state
|
||||
- `siteStore`: Selected site/sector
|
||||
- `aiRequestLogsStore`: AI request/response logs
|
||||
- `pageSizeStore`: Table page size preference
|
||||
|
||||
**Local State**: React `useState` for component-specific state
|
||||
|
||||
### Routing
|
||||
|
||||
**Structure**: React Router v6 with nested routes
|
||||
|
||||
**Routes**:
|
||||
- `/` - Home/Dashboard
|
||||
- `/planner` - Planner Dashboard
|
||||
- `/planner/keywords` - Keywords page
|
||||
- `/planner/clusters` - Clusters page
|
||||
- `/planner/ideas` - Ideas page
|
||||
- `/writer` - Writer Dashboard
|
||||
- `/writer/tasks` - Tasks page
|
||||
- `/settings` - Settings pages
|
||||
|
||||
**Protected Routes**: All routes except `/signin` and `/signup` require authentication
|
||||
|
||||
---
|
||||
|
||||
## Backend Architecture
|
||||
|
||||
### Model Inheritance Hierarchy
|
||||
|
||||
```
|
||||
models.Model
|
||||
└── AccountBaseModel (adds account ForeignKey)
|
||||
└── SiteSectorBaseModel (adds site, sector ForeignKeys)
|
||||
└── Keywords, Clusters, ContentIdeas, Tasks, etc.
|
||||
```
|
||||
|
||||
### ViewSet Inheritance Hierarchy
|
||||
|
||||
```
|
||||
viewsets.ModelViewSet
|
||||
└── AccountModelViewSet (adds account filtering)
|
||||
└── SiteSectorModelViewSet (adds site/sector filtering)
|
||||
└── KeywordViewSet, ClusterViewSet, TasksViewSet, etc.
|
||||
```
|
||||
|
||||
### Middleware Stack
|
||||
|
||||
1. **SecurityMiddleware**: Django security middleware
|
||||
2. **SessionMiddleware**: Session management
|
||||
3. **AuthenticationMiddleware**: User authentication
|
||||
4. **AccountContextMiddleware**: Sets `request.account` from JWT
|
||||
5. **ResourceTrackerMiddleware**: Tracks API request metrics
|
||||
|
||||
### Celery Task Architecture
|
||||
|
||||
**Broker**: Redis
|
||||
|
||||
**Workers**: Separate Celery worker processes
|
||||
|
||||
**Task Structure**:
|
||||
```python
|
||||
@shared_task(bind=True)
|
||||
def my_task(self, ...):
|
||||
# Update progress
|
||||
self.update_state(state='PROGRESS', meta={...})
|
||||
# Do work
|
||||
result = do_work()
|
||||
# Return result
|
||||
return result
|
||||
```
|
||||
|
||||
**Progress Tracking**:
|
||||
- Frontend polls `/api/v1/system/settings/task_progress/{task_id}/`
|
||||
- Backend returns task state and meta information
|
||||
- Progress displayed in modal
|
||||
|
||||
---
|
||||
|
||||
## Database Architecture
|
||||
|
||||
### Core Tables
|
||||
|
||||
- `igny8_accounts`: Account information
|
||||
- `igny8_users`: User accounts
|
||||
- `igny8_plans`: Subscription plans
|
||||
- `igny8_subscriptions`: Active subscriptions
|
||||
- `igny8_sites`: Sites within accounts
|
||||
- `igny8_sectors`: Sectors within sites
|
||||
- `igny8_industries`: Global industry templates
|
||||
- `igny8_industry_sectors`: Industry sector templates
|
||||
|
||||
### Planner Tables
|
||||
|
||||
- `igny8_keywords`: Keywords
|
||||
- `igny8_clusters`: Keyword clusters
|
||||
- `igny8_content_ideas`: Content ideas
|
||||
|
||||
### Writer Tables
|
||||
|
||||
- `igny8_tasks`: Content generation tasks
|
||||
- `igny8_content`: Generated content
|
||||
- `igny8_images`: Generated images
|
||||
|
||||
### System Tables
|
||||
|
||||
- `igny8_ai_prompts`: AI prompt templates
|
||||
- `igny8_integration_settings`: API keys and configuration
|
||||
- `igny8_author_profiles`: Writing style profiles
|
||||
- `igny8_strategies`: Content strategies
|
||||
|
||||
### Billing Tables
|
||||
|
||||
- `igny8_credit_transactions`: Credit transactions
|
||||
- `igny8_usage_logs`: Usage tracking
|
||||
|
||||
### Indexes
|
||||
|
||||
**Account Isolation**: All tables have indexes on `account`
|
||||
|
||||
**Site/Sector Filtering**: Tables with site/sector have composite indexes on `(account, site, sector)`
|
||||
|
||||
**Performance**: Indexes on frequently queried fields (status, created_at, etc.)
|
||||
|
||||
---
|
||||
|
||||
## Security Architecture
|
||||
|
||||
### Authentication
|
||||
|
||||
**JWT Tokens**:
|
||||
- Signed with secret key
|
||||
- Contains user ID and account ID
|
||||
- Expires after configured time
|
||||
- Stored in localStorage (frontend)
|
||||
|
||||
**Session Auth**:
|
||||
- Fallback for admin interface
|
||||
- Django session framework
|
||||
|
||||
### Authorization
|
||||
|
||||
**Role-Based Access Control (RBAC)**:
|
||||
- `developer`: Full system access
|
||||
- `owner`: Full account access
|
||||
- `admin`: Account admin access
|
||||
- `editor`: Content editing access
|
||||
- `viewer`: Read-only access
|
||||
|
||||
**Access Control**:
|
||||
- Account-level: Automatic filtering by `request.account`
|
||||
- Site-level: Filtering by `user.get_accessible_sites()`
|
||||
- Action-level: Permission checks in ViewSet actions
|
||||
|
||||
### Data Isolation
|
||||
|
||||
**Account Isolation**:
|
||||
- All queries filtered by account
|
||||
- Admin/Developer override for system accounts
|
||||
|
||||
**Site Access Control**:
|
||||
- Users can only access granted sites
|
||||
- Admin/Developer override for all sites
|
||||
|
||||
### API Security
|
||||
|
||||
**CORS**: Configured for frontend domain
|
||||
|
||||
**CSRF**: Enabled for session-based auth
|
||||
|
||||
**Rate Limiting**: (Future implementation)
|
||||
|
||||
**Input Validation**: DRF serializers validate all input
|
||||
|
||||
---
|
||||
|
||||
## Deployment Architecture
|
||||
|
||||
### Docker Compose Setup
|
||||
|
||||
**Services**:
|
||||
- `backend`: Django application (port 8010/8011)
|
||||
- `frontend`: React application (port 5173/8021)
|
||||
- `db`: PostgreSQL database
|
||||
- `redis`: Redis for Celery broker and caching
|
||||
- `caddy`: Reverse proxy (HTTPS on port 443)
|
||||
- `celery-worker`: Celery worker process
|
||||
- `celery-beat`: Celery beat scheduler (optional)
|
||||
|
||||
### Environment Configuration
|
||||
|
||||
**Backend**:
|
||||
- `DJANGO_SETTINGS_MODULE`: Django settings module
|
||||
- `DATABASE_URL`: PostgreSQL connection string
|
||||
- `REDIS_URL`: Redis connection string
|
||||
- `SECRET_KEY`: Django secret key
|
||||
- `OPENAI_API_KEY`: OpenAI API key (fallback)
|
||||
|
||||
**Frontend**:
|
||||
- `VITE_API_URL`: Backend API URL
|
||||
- `VITE_APP_NAME`: Application name
|
||||
|
||||
### Scaling Considerations
|
||||
|
||||
**Horizontal Scaling**:
|
||||
- Multiple Celery workers
|
||||
- Multiple backend instances (load balanced)
|
||||
- Multiple frontend instances (static files)
|
||||
|
||||
**Vertical Scaling**:
|
||||
- Database connection pooling
|
||||
- Redis connection pooling
|
||||
- Celery worker concurrency
|
||||
|
||||
### Monitoring
|
||||
|
||||
**Application Monitoring**:
|
||||
- ResourceTrackerMiddleware tracks API request metrics
|
||||
- Celery task monitoring via Flower (optional)
|
||||
|
||||
**Infrastructure Monitoring**:
|
||||
- Portainer for container monitoring
|
||||
- Database monitoring via PostgreSQL logs
|
||||
- Redis monitoring via Redis CLI
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
The IGNY8 architecture is built on:
|
||||
|
||||
1. **Configuration-Driven Design**: Zero duplication, single source of truth
|
||||
2. **Multi-Tenancy Foundation**: Complete account isolation with site/sector hierarchy
|
||||
3. **Template System**: 4 universal templates for all page types
|
||||
4. **Unified AI Interface**: Single AIProcessor for all AI operations
|
||||
5. **Module-Based Organization**: Clear boundaries with shared utilities
|
||||
6. **RESTful API**: Consistent API design with DRF
|
||||
7. **Modern Frontend**: React + TypeScript with Zustand state management
|
||||
8. **Scalable Backend**: Django + Celery for async processing
|
||||
9. **Security First**: JWT auth, RBAC, data isolation
|
||||
10. **Docker Deployment**: Containerized for easy deployment and scaling
|
||||
|
||||
This architecture ensures scalability, maintainability, and extensibility while maintaining a clean separation of concerns across modules.
|
||||
|
||||
776
docs/02-FRONTEND.md
Normal file
776
docs/02-FRONTEND.md
Normal file
@@ -0,0 +1,776 @@
|
||||
# IGNY8 Frontend Documentation
|
||||
|
||||
**Version:** 1.0
|
||||
**Last Updated:** 2025-01-XX
|
||||
**Purpose:** Complete frontend documentation covering pages, components, templates, routing, state management, and configuration system.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Frontend Overview](#frontend-overview)
|
||||
2. [Tech Stack](#tech-stack)
|
||||
3. [Project Structure](#project-structure)
|
||||
4. [Routing System](#routing-system)
|
||||
5. [Template System](#template-system)
|
||||
6. [Component Library](#component-library)
|
||||
7. [State Management](#state-management)
|
||||
8. [API Integration](#api-integration)
|
||||
9. [Configuration System](#configuration-system)
|
||||
10. [Pages](#pages)
|
||||
11. [Hooks](#hooks)
|
||||
12. [Utilities](#utilities)
|
||||
|
||||
---
|
||||
|
||||
## Frontend Overview
|
||||
|
||||
The IGNY8 frontend is a React 19 application built with TypeScript, using Vite as the build tool and Tailwind CSS for styling. The frontend follows a configuration-driven architecture where UI components are rendered from configuration objects, eliminating code duplication and ensuring consistency.
|
||||
|
||||
### Key Features
|
||||
|
||||
- **Configuration-Driven UI**: All tables, filters, and forms driven by config files
|
||||
- **4 Universal Templates**: DashboardTemplate, TablePageTemplate, FormPageTemplate, SystemPageTemplate
|
||||
- **TypeScript**: Full type safety across the application
|
||||
- **Zustand State Management**: Lightweight, performant state management
|
||||
- **React Router v6**: Modern routing with nested routes
|
||||
- **Responsive Design**: Mobile-first approach with Tailwind CSS
|
||||
|
||||
---
|
||||
|
||||
## Tech Stack
|
||||
|
||||
### Core Technologies
|
||||
|
||||
- **React 19**: UI library
|
||||
- **TypeScript**: Type safety
|
||||
- **Vite**: Build tool and dev server
|
||||
- **Tailwind CSS**: Utility-first CSS framework
|
||||
- **React Router v6**: Client-side routing
|
||||
|
||||
### State Management
|
||||
|
||||
- **Zustand**: Lightweight state management library
|
||||
- **localStorage**: Persistence for Zustand stores
|
||||
|
||||
### HTTP Client
|
||||
|
||||
- **Fetch API**: Native browser API with custom wrapper
|
||||
- **Auto-retry**: Automatic retry on network failures
|
||||
- **Token refresh**: Automatic JWT token refresh
|
||||
|
||||
### UI Components
|
||||
|
||||
- **Custom Component Library**: Built-in components (Button, Card, Modal, etc.)
|
||||
- **Icons**: Custom icon library
|
||||
- **Toast Notifications**: Toast notification system
|
||||
|
||||
---
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
frontend/src/
|
||||
├── pages/ # Page components
|
||||
│ ├── Planner/ # Planner module pages
|
||||
│ │ ├── Dashboard.tsx
|
||||
│ │ ├── Keywords.tsx
|
||||
│ │ ├── Clusters.tsx
|
||||
│ │ └── Ideas.tsx
|
||||
│ ├── Writer/ # Writer module pages
|
||||
│ │ ├── Dashboard.tsx
|
||||
│ │ ├── Tasks.tsx
|
||||
│ │ ├── Content.tsx
|
||||
│ │ ├── Drafts.tsx
|
||||
│ │ ├── Images.tsx
|
||||
│ │ └── Published.tsx
|
||||
│ ├── Settings/ # Settings pages
|
||||
│ │ ├── General.tsx
|
||||
│ │ ├── Users.tsx
|
||||
│ │ ├── Sites.tsx
|
||||
│ │ ├── Integration.tsx
|
||||
│ │ └── ...
|
||||
│ ├── AuthPages/ # Authentication pages
|
||||
│ │ ├── SignIn.tsx
|
||||
│ │ └── SignUp.tsx
|
||||
│ └── ...
|
||||
├── templates/ # 4 universal templates
|
||||
│ ├── DashboardTemplate.tsx
|
||||
│ ├── TablePageTemplate.tsx
|
||||
│ ├── FormPageTemplate.tsx
|
||||
│ └── SystemPageTemplate.tsx
|
||||
├── components/ # UI components
|
||||
│ ├── layout/ # Layout components
|
||||
│ │ ├── AppLayout.tsx
|
||||
│ │ ├── Sidebar.tsx
|
||||
│ │ └── Header.tsx
|
||||
│ ├── table/ # Table components
|
||||
│ │ ├── DataTable.tsx
|
||||
│ │ ├── Filters.tsx
|
||||
│ │ └── Pagination.tsx
|
||||
│ ├── ui/ # UI primitives
|
||||
│ │ ├── button/
|
||||
│ │ ├── card/
|
||||
│ │ ├── modal/
|
||||
│ │ └── ...
|
||||
│ └── auth/ # Auth components
|
||||
│ └── ProtectedRoute.tsx
|
||||
├── config/ # Configuration files
|
||||
│ ├── pages/ # Page-specific configs
|
||||
│ │ └── keywords.config.tsx
|
||||
│ ├── snippets/ # Shared snippets
|
||||
│ │ ├── columns.snippets.ts
|
||||
│ │ ├── filters.snippets.ts
|
||||
│ │ └── actions.snippets.ts
|
||||
│ └── routes.config.ts # Route configuration
|
||||
├── store/ # Zustand stores
|
||||
│ ├── authStore.ts # Authentication state
|
||||
│ ├── plannerStore.ts # Planner module state
|
||||
│ ├── siteStore.ts # Site selection state
|
||||
│ ├── sectorStore.ts # Sector selection state
|
||||
│ ├── aiRequestLogsStore.ts # AI request/response logs
|
||||
│ └── pageSizeStore.ts # Table page size preference
|
||||
├── services/ # API clients
|
||||
│ └── api.ts # fetchAPI, API functions
|
||||
├── hooks/ # Custom React hooks
|
||||
│ ├── useProgressModal.ts # Progress modal for long-running tasks
|
||||
│ └── useAuth.ts # Authentication hook
|
||||
├── layout/ # Layout components
|
||||
│ └── AppLayout.tsx # Main app layout wrapper
|
||||
├── utils/ # Utility functions
|
||||
│ └── difficulty.ts # Difficulty utilities
|
||||
├── App.tsx # Root component with routing
|
||||
└── main.tsx # Entry point
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Routing System
|
||||
|
||||
### Route Structure
|
||||
|
||||
**File**: `App.tsx`
|
||||
|
||||
**Public Routes**:
|
||||
- `/signin` - Sign in page
|
||||
- `/signup` - Sign up page
|
||||
|
||||
**Protected Routes** (require authentication):
|
||||
- `/` - Home/Dashboard
|
||||
- `/planner` - Planner Dashboard
|
||||
- `/planner/keywords` - Keywords page
|
||||
- `/planner/clusters` - Clusters page
|
||||
- `/planner/ideas` - Ideas page
|
||||
- `/writer` - Writer Dashboard
|
||||
- `/writer/tasks` - Tasks page
|
||||
- `/writer/content` - Content page
|
||||
- `/writer/drafts` - Drafts page
|
||||
- `/writer/images` - Images page
|
||||
- `/writer/published` - Published page
|
||||
- `/thinker` - Thinker Dashboard
|
||||
- `/thinker/prompts` - Prompts page
|
||||
- `/thinker/author-profiles` - Author Profiles page
|
||||
- `/thinker/strategies` - Strategies page
|
||||
- `/thinker/image-testing` - Image Testing page
|
||||
- `/billing/credits` - Credits page
|
||||
- `/billing/transactions` - Transactions page
|
||||
- `/billing/usage` - Usage page
|
||||
- `/settings` - Settings pages
|
||||
- `/analytics` - Analytics page
|
||||
- `/schedules` - Schedules page
|
||||
|
||||
### Route Configuration
|
||||
|
||||
**File**: `config/routes.config.ts`
|
||||
|
||||
Defines route metadata including:
|
||||
- Route paths
|
||||
- Labels for navigation
|
||||
- Icons
|
||||
- Breadcrumb labels
|
||||
- Nested route structure
|
||||
|
||||
### Protected Routes
|
||||
|
||||
**Component**: `components/auth/ProtectedRoute.tsx`
|
||||
|
||||
**Functionality**:
|
||||
- Checks authentication status from `authStore`
|
||||
- Redirects to `/signin` if not authenticated
|
||||
- Wraps protected routes with `AppLayout`
|
||||
|
||||
---
|
||||
|
||||
## Template System
|
||||
|
||||
### 1. DashboardTemplate
|
||||
|
||||
**Purpose**: Module home pages with KPIs, workflow steps, and charts.
|
||||
|
||||
**Usage**:
|
||||
```tsx
|
||||
<DashboardTemplate
|
||||
title="Planner Dashboard"
|
||||
subtitle="Manage your SEO keywords and content planning"
|
||||
metrics={metrics}
|
||||
workflowSteps={steps}
|
||||
charts={charts}
|
||||
/>
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- Header metrics (KPIs)
|
||||
- Workflow steps
|
||||
- Charts and visualizations
|
||||
- Quick actions
|
||||
|
||||
### 2. TablePageTemplate
|
||||
|
||||
**Purpose**: CRUD table pages (Keywords, Clusters, Tasks, etc.).
|
||||
|
||||
**Usage**:
|
||||
```tsx
|
||||
<TablePageTemplate
|
||||
title="Keywords"
|
||||
subtitle="Manage and organize SEO keywords"
|
||||
columns={columns}
|
||||
data={data}
|
||||
filters={filterConfig}
|
||||
onSort={(field, direction) => ...}
|
||||
onEdit={(row) => ...}
|
||||
onDelete={(row) => ...}
|
||||
bulkActions={bulkActions}
|
||||
/>
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- Data table with sorting
|
||||
- Filters (text, select, date range, custom)
|
||||
- Pagination
|
||||
- Bulk actions
|
||||
- Row actions (edit, delete)
|
||||
- AI Request/Response Logs section
|
||||
- Import/Export functionality
|
||||
|
||||
**Configuration**:
|
||||
- `columns`: Column definitions (key, label, sortable, render, etc.)
|
||||
- `filters`: Filter definitions (type, options, custom render)
|
||||
- `bulkActions`: Bulk action definitions
|
||||
- `actions`: Row action definitions
|
||||
|
||||
### 3. FormPageTemplate
|
||||
|
||||
**Purpose**: Settings/form pages (Settings, Integration, etc.).
|
||||
|
||||
**Usage**:
|
||||
```tsx
|
||||
<FormPageTemplate
|
||||
title="Settings"
|
||||
subtitle="Configure your account settings"
|
||||
sections={sections}
|
||||
onSubmit={(data) => ...}
|
||||
/>
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- Form sections
|
||||
- Form validation
|
||||
- Save/Cancel buttons
|
||||
- Success/Error notifications
|
||||
|
||||
### 4. SystemPageTemplate
|
||||
|
||||
**Purpose**: System/admin pages (Logs, Status, Monitoring).
|
||||
|
||||
**Usage**:
|
||||
```tsx
|
||||
<SystemPageTemplate
|
||||
title="System Status"
|
||||
subtitle="Monitor system health and performance"
|
||||
content={content}
|
||||
/>
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- System information display
|
||||
- Logs viewer
|
||||
- Status indicators
|
||||
- Performance metrics
|
||||
|
||||
---
|
||||
|
||||
## Component Library
|
||||
|
||||
### Layout Components
|
||||
|
||||
#### AppLayout
|
||||
**File**: `layout/AppLayout.tsx`
|
||||
|
||||
**Purpose**: Main app layout wrapper.
|
||||
|
||||
**Features**:
|
||||
- Sidebar navigation
|
||||
- Header with user menu
|
||||
- Main content area
|
||||
- Breadcrumbs
|
||||
|
||||
#### Sidebar
|
||||
**File**: `components/layout/Sidebar.tsx`
|
||||
|
||||
**Purpose**: Navigation sidebar.
|
||||
|
||||
**Features**:
|
||||
- Module navigation
|
||||
- Active route highlighting
|
||||
- Collapsible sections
|
||||
|
||||
#### Header
|
||||
**File**: `components/layout/Header.tsx`
|
||||
|
||||
**Purpose**: Top header bar.
|
||||
|
||||
**Features**:
|
||||
- User menu
|
||||
- Notifications
|
||||
- Site/Sector selector
|
||||
|
||||
### Table Components
|
||||
|
||||
#### DataTable
|
||||
**File**: `components/table/DataTable.tsx`
|
||||
|
||||
**Purpose**: Data table component.
|
||||
|
||||
**Features**:
|
||||
- Sortable columns
|
||||
- Selectable rows
|
||||
- Row actions
|
||||
- Responsive design
|
||||
|
||||
#### Filters
|
||||
**File**: `components/table/Filters.tsx`
|
||||
|
||||
**Purpose**: Filter component.
|
||||
|
||||
**Features**:
|
||||
- Text filters
|
||||
- Select filters
|
||||
- Date range filters
|
||||
- Custom filters
|
||||
|
||||
#### Pagination
|
||||
**File**: `components/ui/pagination/CompactPagination.tsx`
|
||||
|
||||
**Purpose**: Pagination component.
|
||||
|
||||
**Features**:
|
||||
- Page navigation
|
||||
- Page size selector
|
||||
- Total count display
|
||||
|
||||
### UI Components
|
||||
|
||||
#### Button
|
||||
**File**: `components/ui/button/Button.tsx`
|
||||
|
||||
**Variants**: primary, secondary, danger, ghost, link
|
||||
|
||||
**Sizes**: sm, md, lg
|
||||
|
||||
#### Card
|
||||
**File**: `components/ui/card/Card.tsx`
|
||||
|
||||
**Purpose**: Card container component.
|
||||
|
||||
#### Modal
|
||||
**File**: `components/ui/modal/Modal.tsx`
|
||||
|
||||
**Purpose**: Modal dialog component.
|
||||
|
||||
**Variants**: FormModal, ProgressModal, AlertModal
|
||||
|
||||
#### Toast
|
||||
**File**: `components/ui/toast/ToastContainer.tsx`
|
||||
|
||||
**Purpose**: Toast notification system.
|
||||
|
||||
**Types**: success, error, warning, info
|
||||
|
||||
#### Input
|
||||
**File**: `components/form/input/InputField.tsx`
|
||||
|
||||
**Purpose**: Text input component.
|
||||
|
||||
#### Select
|
||||
**File**: `components/form/SelectDropdown.tsx`
|
||||
|
||||
**Purpose**: Select dropdown component.
|
||||
|
||||
#### Checkbox
|
||||
**File**: `components/form/input/Checkbox.tsx`
|
||||
|
||||
**Purpose**: Checkbox component.
|
||||
|
||||
### Auth Components
|
||||
|
||||
#### ProtectedRoute
|
||||
**File**: `components/auth/ProtectedRoute.tsx`
|
||||
|
||||
**Purpose**: Route protection component.
|
||||
|
||||
**Functionality**:
|
||||
- Checks authentication
|
||||
- Redirects to signin if not authenticated
|
||||
- Wraps children with AppLayout
|
||||
|
||||
---
|
||||
|
||||
## State Management
|
||||
|
||||
### Zustand Stores
|
||||
|
||||
#### authStore
|
||||
**File**: `store/authStore.ts`
|
||||
|
||||
**State**:
|
||||
- `user`: Current user object
|
||||
- `token`: JWT access token
|
||||
- `refreshToken`: JWT refresh token
|
||||
- `isAuthenticated`: Authentication status
|
||||
- `loading`: Loading state
|
||||
|
||||
**Actions**:
|
||||
- `login(email, password)`: Sign in user
|
||||
- `logout()`: Sign out user
|
||||
- `register(data)`: Register new user
|
||||
- `setUser(user)`: Set user object
|
||||
- `setToken(token)`: Set access token
|
||||
- `refreshToken()`: Refresh access token
|
||||
|
||||
**Persistence**: localStorage (persisted)
|
||||
|
||||
#### plannerStore
|
||||
**File**: `store/plannerStore.ts`
|
||||
|
||||
**State**: Planner module-specific state
|
||||
|
||||
**Actions**: Planner module actions
|
||||
|
||||
#### siteStore
|
||||
**File**: `store/siteStore.ts`
|
||||
|
||||
**State**:
|
||||
- `activeSite`: Currently selected site
|
||||
- `sites`: List of accessible sites
|
||||
|
||||
**Actions**:
|
||||
- `setActiveSite(site)`: Set active site
|
||||
- `loadSites()`: Load accessible sites
|
||||
|
||||
**Persistence**: localStorage (persisted)
|
||||
|
||||
#### sectorStore
|
||||
**File**: `store/sectorStore.ts`
|
||||
|
||||
**State**:
|
||||
- `activeSector`: Currently selected sector
|
||||
- `sectors`: List of sectors for active site
|
||||
|
||||
**Actions**:
|
||||
- `setActiveSector(sector)`: Set active sector
|
||||
- `loadSectorsForSite(siteId)`: Load sectors for site
|
||||
|
||||
**Persistence**: localStorage (persisted)
|
||||
|
||||
#### aiRequestLogsStore
|
||||
**File**: `store/aiRequestLogsStore.ts`
|
||||
|
||||
**State**:
|
||||
- `logs`: Array of AI request/response logs
|
||||
|
||||
**Actions**:
|
||||
- `addLog(log)`: Add new log entry
|
||||
- `addRequestStep(logId, step)`: Add request step to log
|
||||
- `addResponseStep(logId, step)`: Add response step to log
|
||||
- `updateLog(logId, data)`: Update log entry
|
||||
- `clearLogs()`: Clear all logs
|
||||
|
||||
**Purpose**: Tracks AI function execution with step-by-step logs
|
||||
|
||||
#### pageSizeStore
|
||||
**File**: `store/pageSizeStore.ts`
|
||||
|
||||
**State**:
|
||||
- `pageSize`: Table page size preference
|
||||
|
||||
**Actions**:
|
||||
- `setPageSize(size)`: Set page size
|
||||
|
||||
**Persistence**: localStorage (persisted)
|
||||
|
||||
---
|
||||
|
||||
## API Integration
|
||||
|
||||
### API Service
|
||||
|
||||
**File**: `services/api.ts`
|
||||
|
||||
**Functions**:
|
||||
- `fetchAPI(url, options)`: Generic API fetch wrapper
|
||||
- `fetchKeywords(filters)`: Fetch keywords
|
||||
- `createKeyword(data)`: Create keyword
|
||||
- `updateKeyword(id, data)`: Update keyword
|
||||
- `deleteKeyword(id)`: Delete keyword
|
||||
- `bulkDeleteKeywords(ids)`: Bulk delete keywords
|
||||
- `autoClusterKeywords(ids)`: Auto-cluster keywords
|
||||
- `fetchClusters(filters)`: Fetch clusters
|
||||
- `autoGenerateIdeas(clusterIds)`: Auto-generate ideas
|
||||
- `fetchTasks(filters)`: Fetch tasks
|
||||
- `autoGenerateContent(taskIds)`: Auto-generate content
|
||||
- `autoGenerateImages(taskIds)`: Auto-generate images
|
||||
- And more...
|
||||
|
||||
**Features**:
|
||||
- Automatic JWT token inclusion
|
||||
- Automatic token refresh on 401
|
||||
- Auto-retry on network failures
|
||||
- Error handling
|
||||
- Request/response logging
|
||||
|
||||
### API Base URL
|
||||
|
||||
**Auto-detection**:
|
||||
- Checks environment variables (`VITE_BACKEND_URL`, `VITE_API_URL`)
|
||||
- Falls back to auto-detection based on current origin
|
||||
- Supports localhost, IP addresses, and production subdomain
|
||||
|
||||
**Default**: `https://api.igny8.com/api`
|
||||
|
||||
---
|
||||
|
||||
## Configuration System
|
||||
|
||||
### Page-Local Config
|
||||
|
||||
**Location**: `config/pages/`
|
||||
|
||||
**Example**: `keywords.config.tsx`
|
||||
|
||||
**Structure**:
|
||||
```tsx
|
||||
export const createKeywordsPageConfig = () => ({
|
||||
columns: [...],
|
||||
filters: [...],
|
||||
bulkActions: [...],
|
||||
actions: [...],
|
||||
});
|
||||
```
|
||||
|
||||
**Usage**:
|
||||
```tsx
|
||||
import { createKeywordsPageConfig } from '../../config/pages/keywords.config';
|
||||
|
||||
const config = createKeywordsPageConfig();
|
||||
```
|
||||
|
||||
### Shared Snippets
|
||||
|
||||
**Location**: `config/snippets/`
|
||||
|
||||
#### columns.snippets.ts
|
||||
**Purpose**: Reusable column definitions.
|
||||
|
||||
**Examples**:
|
||||
- `statusColumn`: Status column with badge
|
||||
- `titleColumn`: Title column with link
|
||||
- `dateColumn`: Date column with formatting
|
||||
|
||||
#### filters.snippets.ts
|
||||
**Purpose**: Reusable filter definitions.
|
||||
|
||||
**Examples**:
|
||||
- `statusFilter`: Status dropdown filter
|
||||
- `dateRangeFilter`: Date range filter
|
||||
- `searchFilter`: Text search filter
|
||||
|
||||
#### actions.snippets.ts
|
||||
**Purpose**: Reusable action definitions.
|
||||
|
||||
**Examples**:
|
||||
- `commonActions`: Edit, Delete actions
|
||||
- `bulkActions`: Bulk delete, bulk update actions
|
||||
|
||||
### Route Configuration
|
||||
|
||||
**File**: `config/routes.config.ts`
|
||||
|
||||
**Structure**:
|
||||
```tsx
|
||||
export const routes: RouteConfig[] = [
|
||||
{
|
||||
path: '/planner',
|
||||
label: 'Planner',
|
||||
icon: 'Planner',
|
||||
children: [
|
||||
{ path: '/planner/keywords', label: 'Keywords' },
|
||||
...
|
||||
],
|
||||
},
|
||||
...
|
||||
];
|
||||
```
|
||||
|
||||
**Functions**:
|
||||
- `getBreadcrumbs(pathname)`: Get breadcrumbs for current route
|
||||
|
||||
---
|
||||
|
||||
## Pages
|
||||
|
||||
### Planner Module
|
||||
|
||||
#### Keywords Page
|
||||
**File**: `pages/Planner/Keywords.tsx`
|
||||
|
||||
**Features**:
|
||||
- Keyword CRUD operations
|
||||
- Auto-cluster functionality
|
||||
- Import/Export (CSV)
|
||||
- Filters (status, cluster, intent, difficulty, volume)
|
||||
- Bulk actions (delete, status update)
|
||||
- AI Request/Response Logs
|
||||
|
||||
**Configuration**: Uses `keywords.config.tsx`
|
||||
|
||||
#### Clusters Page
|
||||
**File**: `pages/Planner/Clusters.tsx`
|
||||
|
||||
**Features**:
|
||||
- Cluster CRUD operations
|
||||
- Auto-generate ideas functionality
|
||||
- Filters (status, sector)
|
||||
- Bulk actions
|
||||
|
||||
#### Ideas Page
|
||||
**File**: `pages/Planner/Ideas.tsx`
|
||||
|
||||
**Features**:
|
||||
- Content ideas CRUD operations
|
||||
- Filters (status, cluster, content type)
|
||||
- Bulk actions
|
||||
|
||||
### Writer Module
|
||||
|
||||
#### Tasks Page
|
||||
**File**: `pages/Writer/Tasks.tsx`
|
||||
|
||||
**Features**:
|
||||
- Task CRUD operations
|
||||
- Auto-generate content functionality
|
||||
- Auto-generate images functionality
|
||||
- Filters (status, cluster, content type)
|
||||
- Bulk actions
|
||||
|
||||
### Settings Pages
|
||||
|
||||
#### Sites Page
|
||||
**File**: `pages/Settings/Sites.tsx`
|
||||
|
||||
**Features**:
|
||||
- Site CRUD operations
|
||||
- Site activation/deactivation
|
||||
- Multiple sites can be active simultaneously
|
||||
|
||||
#### Integration Page
|
||||
**File**: `pages/Settings/Integration.tsx`
|
||||
|
||||
**Features**:
|
||||
- Integration settings (OpenAI, Runware)
|
||||
- API key configuration
|
||||
- Test connections
|
||||
- Image generation testing
|
||||
|
||||
---
|
||||
|
||||
## Hooks
|
||||
|
||||
### useProgressModal
|
||||
|
||||
**File**: `hooks/useProgressModal.ts`
|
||||
|
||||
**Purpose**: Progress modal for long-running Celery tasks.
|
||||
|
||||
**Usage**:
|
||||
```tsx
|
||||
const progressModal = useProgressModal();
|
||||
|
||||
// Start task
|
||||
progressModal.start(taskId, 'Task started...');
|
||||
|
||||
// Task progress is automatically polled
|
||||
// Modal displays progress updates
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- Automatic polling of task progress
|
||||
- Progress percentage display
|
||||
- Step-by-step logs (request/response steps)
|
||||
- Success/Error handling
|
||||
- Auto-close on completion
|
||||
|
||||
### useAuth
|
||||
|
||||
**File**: `hooks/useAuth.ts`
|
||||
|
||||
**Purpose**: Authentication hook.
|
||||
|
||||
**Usage**:
|
||||
```tsx
|
||||
const { user, isAuthenticated, login, logout } = useAuth();
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- Access to auth state
|
||||
- Login/logout functions
|
||||
- Authentication status
|
||||
|
||||
---
|
||||
|
||||
## Utilities
|
||||
|
||||
### Difficulty Utilities
|
||||
|
||||
**File**: `utils/difficulty.ts`
|
||||
|
||||
**Functions**:
|
||||
- `getDifficultyLabelFromNumber(number)`: Get difficulty label (Easy, Medium, Hard)
|
||||
- `getDifficultyRange()`: Get difficulty range options
|
||||
|
||||
### API Utilities
|
||||
|
||||
**File**: `services/api.ts`
|
||||
|
||||
**Helper Functions**:
|
||||
- `getActiveSiteId()`: Get active site ID from store
|
||||
- `getActiveSectorId()`: Get active sector ID from store
|
||||
- `getAuthToken()`: Get JWT token from store
|
||||
- `getRefreshToken()`: Get refresh token from store
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
The IGNY8 frontend is built on:
|
||||
|
||||
1. **Configuration-Driven Architecture**: Zero duplication, single source of truth
|
||||
2. **4 Universal Templates**: Reusable templates for all page types
|
||||
3. **TypeScript**: Full type safety
|
||||
4. **Zustand State Management**: Lightweight, performant state management
|
||||
5. **React Router v6**: Modern routing with nested routes
|
||||
6. **Component Library**: Reusable UI components
|
||||
7. **API Integration**: Automatic token handling and error management
|
||||
8. **Responsive Design**: Mobile-first with Tailwind CSS
|
||||
|
||||
This architecture ensures consistency, maintainability, and extensibility while providing a great user experience.
|
||||
|
||||
860
docs/03-BACKEND.md
Normal file
860
docs/03-BACKEND.md
Normal file
@@ -0,0 +1,860 @@
|
||||
# IGNY8 Backend Documentation
|
||||
|
||||
**Version:** 1.0
|
||||
**Last Updated:** 2025-01-XX
|
||||
**Purpose:** Complete backend documentation covering models, views, APIs, modules, serializers, tasks, and structure.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Backend Overview](#backend-overview)
|
||||
2. [Tech Stack](#tech-stack)
|
||||
3. [Project Structure](#project-structure)
|
||||
4. [Models](#models)
|
||||
5. [ViewSets](#viewsets)
|
||||
6. [Serializers](#serializers)
|
||||
7. [Celery Tasks](#celery-tasks)
|
||||
8. [API Endpoints](#api-endpoints)
|
||||
9. [Base Classes](#base-classes)
|
||||
10. [Middleware](#middleware)
|
||||
11. [Utilities](#utilities)
|
||||
12. [Modules](#modules)
|
||||
|
||||
---
|
||||
|
||||
## Backend Overview
|
||||
|
||||
The IGNY8 backend is a Django 5.2+ application using Django REST Framework (DRF) for API endpoints. The backend follows a modular architecture with clear separation of concerns, automatic account isolation, and support for asynchronous task processing via Celery.
|
||||
|
||||
### Key Features
|
||||
|
||||
- **Multi-Tenancy**: Complete account isolation with automatic filtering
|
||||
- **RESTful API**: DRF ViewSets with consistent response format
|
||||
- **Celery Integration**: Asynchronous task processing for long-running operations
|
||||
- **Account/Site/Sector Hierarchy**: Hierarchical data organization
|
||||
- **AI Integration**: Unified AIProcessor for all AI operations
|
||||
- **Progress Tracking**: Real-time progress updates for Celery tasks
|
||||
|
||||
---
|
||||
|
||||
## Tech Stack
|
||||
|
||||
### Core Technologies
|
||||
|
||||
- **Django 5.2+**: Web framework
|
||||
- **Django REST Framework**: API framework
|
||||
- **PostgreSQL**: Database
|
||||
- **Celery**: Asynchronous task queue
|
||||
- **Redis**: Celery broker and caching
|
||||
|
||||
### Key Libraries
|
||||
|
||||
- **django-filter**: Advanced filtering
|
||||
- **djangorestframework-simplejwt**: JWT authentication
|
||||
- **requests**: HTTP client for external APIs
|
||||
- **python-dotenv**: Environment variable management
|
||||
|
||||
---
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
backend/igny8_core/
|
||||
├── auth/ # Multi-tenancy and authentication
|
||||
│ ├── models.py # Account, User, Plan, Site, Sector, Industry models
|
||||
│ ├── views.py # Account, User, Site, Sector ViewSets
|
||||
│ ├── serializers.py # Account, User, Plan serializers
|
||||
│ └── urls.py # Auth module URLs
|
||||
├── modules/ # Feature modules
|
||||
│ ├── planner/ # Keywords, Clusters, Ideas
|
||||
│ │ ├── models.py # Keywords, Clusters, ContentIdeas models
|
||||
│ │ ├── views.py # KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
|
||||
│ │ ├── tasks.py # Celery tasks for AI operations
|
||||
│ │ ├── serializers.py # Model serializers
|
||||
│ │ └── urls.py # Planner module URLs
|
||||
│ ├── writer/ # Tasks, Content, Images
|
||||
│ │ ├── models.py # Tasks, Content, Images models
|
||||
│ │ ├── views.py # TasksViewSet
|
||||
│ │ ├── tasks.py # Celery tasks for content/image generation
|
||||
│ │ └── urls.py # Writer module URLs
|
||||
│ ├── system/ # Settings, Prompts, Integration
|
||||
│ │ ├── models.py # AIPrompt, IntegrationSettings, AuthorProfile, Strategy
|
||||
│ │ ├── views.py # AIPromptViewSet, AuthorProfileViewSet
|
||||
│ │ ├── integration_views.py # IntegrationSettingsViewSet, task_progress
|
||||
│ │ ├── utils.py # Default prompts, prompt loading
|
||||
│ │ └── urls.py # System module URLs
|
||||
│ └── billing/ # Credits, Transactions, Usage
|
||||
│ ├── models.py # CreditTransaction, UsageLog models
|
||||
│ ├── views.py # Billing ViewSets
|
||||
│ └── services.py # CreditService
|
||||
├── api/ # API base classes
|
||||
│ ├── base.py # AccountModelViewSet, SiteSectorModelViewSet
|
||||
│ └── pagination.py # CustomPageNumberPagination
|
||||
├── utils/ # Shared utilities
|
||||
│ ├── ai_processor.py # Unified AI interface
|
||||
│ └── content_normalizer.py # Content processing utilities
|
||||
├── middleware/ # Custom middleware
|
||||
│ ├── account.py # AccountContextMiddleware (sets request.account)
|
||||
│ └── resource_tracker.py # ResourceTrackerMiddleware (API metrics)
|
||||
├── settings.py # Django settings
|
||||
├── urls.py # Root URL configuration
|
||||
└── celery.py # Celery configuration
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Models
|
||||
|
||||
### Base Models
|
||||
|
||||
#### AccountBaseModel
|
||||
**File**: `auth/models.py`
|
||||
|
||||
**Purpose**: Base model for all account-isolated models.
|
||||
|
||||
**Fields**:
|
||||
- `account`: ForeignKey to Account
|
||||
- `created_at`: DateTimeField (auto_now_add)
|
||||
- `updated_at`: DateTimeField (auto_now)
|
||||
|
||||
**Usage**: All models that need account isolation inherit from this.
|
||||
|
||||
|
||||
#### SiteSectorBaseModel
|
||||
**File**: `auth/models.py`
|
||||
|
||||
**Purpose**: Base model for models that belong to Site and Sector.
|
||||
|
||||
**Fields**:
|
||||
- Inherits from `AccountBaseModel`
|
||||
- `site`: ForeignKey to Site
|
||||
- `sector`: ForeignKey to Sector
|
||||
|
||||
**Methods**:
|
||||
- `save()`: Automatically sets `account` from `site.account` and validates sector belongs to site
|
||||
|
||||
**Usage**: Models like Keywords, Clusters, ContentIdeas, Tasks inherit from this.
|
||||
|
||||
### Auth Models
|
||||
|
||||
#### Account
|
||||
**File**: `auth/models.py`
|
||||
|
||||
**Table**: `igny8_accounts`
|
||||
|
||||
**Fields**:
|
||||
- `name`: CharField
|
||||
- `slug`: SlugField (unique)
|
||||
- `owner`: ForeignKey to User
|
||||
- `stripe_customer_id`: CharField (optional)
|
||||
- `plan`: ForeignKey to Plan
|
||||
- `credits`: IntegerField (default: 0)
|
||||
- `status`: CharField (choices: active, suspended, trial, cancelled)
|
||||
|
||||
**Methods**:
|
||||
- `is_system_account()`: Returns True if account is system account (aws-admin, default-account, default)
|
||||
|
||||
#### User
|
||||
**File**: `auth/models.py`
|
||||
|
||||
**Table**: `igny8_users`
|
||||
|
||||
**Fields**:
|
||||
- Inherits from `AbstractUser`
|
||||
- `email`: EmailField (unique, USERNAME_FIELD)
|
||||
- `account`: ForeignKey to Account
|
||||
- `role`: CharField (choices: developer, owner, admin, editor, viewer, system_bot)
|
||||
|
||||
**Methods**:
|
||||
- `has_role(*roles)`: Check if user has any of the specified roles
|
||||
- `is_owner_or_admin()`: Returns True if role is owner or admin
|
||||
- `is_developer()`: Returns True if role is developer or is_superuser
|
||||
- `is_admin_or_developer()`: Returns True if role is admin or developer
|
||||
- `is_system_account_user()`: Returns True if user belongs to system account
|
||||
- `get_accessible_sites()`: Returns QuerySet of sites user can access
|
||||
|
||||
#### Plan
|
||||
**File**: `auth/models.py`
|
||||
|
||||
**Table**: `igny8_plans`
|
||||
|
||||
**Fields**:
|
||||
- `name`: CharField
|
||||
- `slug`: SlugField (unique)
|
||||
- `price`: DecimalField
|
||||
- `billing_cycle`: CharField (choices: monthly, annual)
|
||||
- `features`: JSONField (array of feature strings)
|
||||
- `max_users`: IntegerField
|
||||
- `max_sites`: IntegerField
|
||||
- `max_keywords`: IntegerField
|
||||
- `max_clusters`: IntegerField
|
||||
- `max_content_ideas`: IntegerField
|
||||
- `daily_cluster_limit`: IntegerField
|
||||
- `monthly_cluster_ai_credits`: IntegerField
|
||||
- `daily_content_tasks`: IntegerField
|
||||
- `daily_ai_requests`: IntegerField
|
||||
- `monthly_word_count_limit`: IntegerField
|
||||
- `monthly_content_ai_credits`: IntegerField
|
||||
- `monthly_image_count`: IntegerField
|
||||
- `daily_image_generation_limit`: IntegerField
|
||||
- `monthly_image_ai_credits`: IntegerField
|
||||
- `max_images_per_task`: IntegerField
|
||||
- `image_model_choices`: JSONField
|
||||
- `included_credits`: IntegerField
|
||||
- `extra_credit_price`: DecimalField
|
||||
- And more...
|
||||
|
||||
**Methods**:
|
||||
- `clean()`: Validates plan limits
|
||||
- `get_effective_credits_per_month()`: Returns included_credits or credits_per_month
|
||||
|
||||
#### Site
|
||||
**File**: `auth/models.py`
|
||||
|
||||
**Table**: `igny8_sites`
|
||||
|
||||
**Fields**:
|
||||
- Inherits from `AccountBaseModel`
|
||||
- `name`: CharField
|
||||
- `slug`: SlugField (unique per account)
|
||||
- `domain`: URLField (optional)
|
||||
- `industry`: ForeignKey to Industry (optional)
|
||||
- `is_active`: BooleanField
|
||||
- `status`: CharField (choices: active, inactive, suspended)
|
||||
- `wp_url`: URLField (optional, WordPress integration)
|
||||
- `wp_username`: CharField (optional)
|
||||
- `wp_app_password`: CharField (optional)
|
||||
|
||||
**Methods**:
|
||||
- `get_active_sectors_count()`: Get count of active sectors
|
||||
- `can_add_sector()`: Check if site can add another sector (max 5)
|
||||
|
||||
#### Sector
|
||||
**File**: `auth/models.py`
|
||||
|
||||
**Table**: `igny8_sectors`
|
||||
|
||||
**Fields**:
|
||||
- Inherits from `AccountBaseModel`
|
||||
- `site`: ForeignKey to Site
|
||||
- `industry_sector`: ForeignKey to IndustrySector (optional, template reference)
|
||||
- `name`: CharField
|
||||
- `slug`: SlugField (unique per site)
|
||||
- `is_active`: BooleanField
|
||||
- `status`: CharField (choices: active, inactive)
|
||||
|
||||
### Planner Models
|
||||
|
||||
#### Keywords
|
||||
**File**: `modules/planner/models.py`
|
||||
|
||||
**Table**: `igny8_keywords`
|
||||
|
||||
**Fields**:
|
||||
- Inherits from `SiteSectorBaseModel`
|
||||
- `keyword`: CharField
|
||||
- `volume`: IntegerField
|
||||
- `difficulty`: IntegerField
|
||||
- `intent`: CharField (choices: informational, navigational, commercial, transactional)
|
||||
- `cluster`: ForeignKey to Clusters (optional)
|
||||
- `status`: CharField (choices: active, pending, archived)
|
||||
|
||||
#### Clusters
|
||||
**File**: `modules/planner/models.py`
|
||||
|
||||
**Table**: `igny8_clusters`
|
||||
|
||||
**Fields**:
|
||||
- Inherits from `SiteSectorBaseModel`
|
||||
- `name`: CharField (unique)
|
||||
- `description`: TextField
|
||||
- `keywords_count`: IntegerField
|
||||
- `volume`: IntegerField
|
||||
- `mapped_pages`: IntegerField
|
||||
- `status`: CharField
|
||||
|
||||
#### ContentIdeas
|
||||
**File**: `modules/planner/models.py`
|
||||
|
||||
**Table**: `igny8_content_ideas`
|
||||
|
||||
**Fields**:
|
||||
- Inherits from `SiteSectorBaseModel`
|
||||
- `idea_title`: CharField
|
||||
- `description`: TextField
|
||||
- `content_structure`: CharField (choices: cluster_hub, landing_page, pillar_page, supporting_page)
|
||||
- `content_type`: CharField (choices: blog_post, article, guide, tutorial)
|
||||
- `target_keywords`: CharField (comma-separated, legacy)
|
||||
- `keyword_objects`: ManyToManyField to Keywords
|
||||
- `keyword_cluster`: ForeignKey to Clusters
|
||||
- `status`: CharField (choices: new, scheduled, published)
|
||||
- `estimated_word_count`: IntegerField
|
||||
|
||||
### Writer Models
|
||||
|
||||
#### Tasks
|
||||
**File**: `modules/writer/models.py`
|
||||
|
||||
**Table**: `igny8_tasks`
|
||||
|
||||
**Fields**:
|
||||
- Inherits from `SiteSectorBaseModel`
|
||||
- `title`: CharField
|
||||
- `description`: TextField
|
||||
- `keywords`: CharField (comma-separated, legacy)
|
||||
- `keyword_objects`: ManyToManyField to Keywords
|
||||
- `cluster`: ForeignKey to Clusters
|
||||
- `idea`: ForeignKey to ContentIdeas
|
||||
- `content_structure`: CharField
|
||||
- `content_type`: CharField
|
||||
- `status`: CharField (choices: queued, in_progress, draft, review, published, completed)
|
||||
- `content`: TextField (generated content)
|
||||
- `word_count`: IntegerField
|
||||
- `meta_title`: CharField
|
||||
- `meta_description`: TextField
|
||||
- `assigned_post_id`: IntegerField (WordPress post ID)
|
||||
- `post_url`: URLField
|
||||
|
||||
#### Content
|
||||
**File**: `modules/writer/models.py`
|
||||
|
||||
**Table**: `igny8_content`
|
||||
|
||||
**Fields**:
|
||||
- Inherits from `SiteSectorBaseModel`
|
||||
- `task`: OneToOneField to Tasks
|
||||
- `html_content`: TextField
|
||||
- `word_count`: IntegerField
|
||||
- `metadata`: JSONField
|
||||
|
||||
**Methods**:
|
||||
- `save()`: Automatically sets account, site, sector from task
|
||||
|
||||
#### Images
|
||||
**File**: `modules/writer/models.py`
|
||||
|
||||
**Table**: `igny8_images`
|
||||
|
||||
**Fields**:
|
||||
- Inherits from `SiteSectorBaseModel`
|
||||
- `task`: ForeignKey to Tasks
|
||||
- `image_type`: CharField (choices: featured, desktop, mobile, in_article)
|
||||
- `image_url`: URLField
|
||||
- `image_path`: CharField (local path)
|
||||
- `prompt`: TextField
|
||||
- `status`: CharField
|
||||
- `position`: IntegerField
|
||||
|
||||
**Methods**:
|
||||
- `save()`: Automatically sets account, site, sector from task
|
||||
|
||||
### System Models
|
||||
|
||||
#### AIPrompt
|
||||
**File**: `modules/system/models.py`
|
||||
|
||||
**Table**: `igny8_ai_prompts`
|
||||
|
||||
**Fields**:
|
||||
- Inherits from `AccountBaseModel`
|
||||
- `prompt_type`: CharField (choices: clustering, ideas, content_generation, image_prompt_extraction, image_prompt_template, negative_prompt)
|
||||
- `prompt_value`: TextField
|
||||
- `default_prompt`: TextField
|
||||
- `is_active`: BooleanField
|
||||
|
||||
**Unique Constraint**: `(account, prompt_type)`
|
||||
|
||||
#### IntegrationSettings
|
||||
**File**: `modules/system/models.py`
|
||||
|
||||
**Table**: `igny8_integration_settings`
|
||||
|
||||
**Fields**:
|
||||
- Inherits from `AccountBaseModel`
|
||||
- `integration_type`: CharField (choices: openai, runware, gsc, image_generation)
|
||||
- `config`: JSONField (API keys, settings, etc.)
|
||||
- `is_active`: BooleanField
|
||||
|
||||
**Unique Constraint**: `(account, integration_type)`
|
||||
|
||||
#### AuthorProfile
|
||||
**File**: `modules/system/models.py`
|
||||
|
||||
**Table**: `igny8_author_profiles`
|
||||
|
||||
**Fields**:
|
||||
- Inherits from `AccountBaseModel`
|
||||
- `name`: CharField
|
||||
- `description`: TextField
|
||||
- `tone`: CharField
|
||||
- `language`: CharField
|
||||
- `structure_template`: JSONField
|
||||
- `is_active`: BooleanField
|
||||
|
||||
#### Strategy
|
||||
**File**: `modules/system/models.py`
|
||||
|
||||
**Table**: `igny8_strategies`
|
||||
|
||||
**Fields**:
|
||||
- Inherits from `AccountBaseModel`
|
||||
- `name`: CharField
|
||||
- `description`: TextField
|
||||
- `sector`: ForeignKey to Sector (optional)
|
||||
- `prompt_types`: JSONField
|
||||
- `section_logic`: JSONField
|
||||
- `is_active`: BooleanField
|
||||
|
||||
---
|
||||
|
||||
## ViewSets
|
||||
|
||||
### Base ViewSets
|
||||
|
||||
#### AccountModelViewSet
|
||||
**File**: `api/base.py`
|
||||
|
||||
**Purpose**: Base ViewSet with automatic account filtering.
|
||||
|
||||
**Methods**:
|
||||
- `get_queryset()`: Filters queryset by `request.account` (with admin/developer override)
|
||||
- `perform_create()`: Sets account on created objects
|
||||
- `get_serializer_context()`: Adds account to serializer context
|
||||
|
||||
**Access Control**:
|
||||
- Admin/Developer users: Bypass account filtering
|
||||
- System account users: Bypass account filtering
|
||||
- Regular users: Only see data from their account
|
||||
|
||||
#### SiteSectorModelViewSet
|
||||
**File**: `api/base.py`
|
||||
|
||||
**Purpose**: Base ViewSet with site/sector filtering and access control.
|
||||
|
||||
**Inherits**: `AccountModelViewSet`
|
||||
|
||||
**Methods**:
|
||||
- `get_queryset()`: Filters by account, accessible sites, and optional site_id/sector_id
|
||||
- `perform_create()`: Validates site access and sector-site relationship
|
||||
- `get_serializer_context()`: Adds accessible sites and sectors to context
|
||||
|
||||
**Access Control**:
|
||||
- Developers: All active sites
|
||||
- System account users: All active sites
|
||||
- Owners/Admins: All sites in their account
|
||||
- Editors/Viewers: Only sites granted via `SiteUserAccess`
|
||||
|
||||
### Planner ViewSets
|
||||
|
||||
#### KeywordViewSet
|
||||
**File**: `modules/planner/views.py`
|
||||
|
||||
**Inherits**: `SiteSectorModelViewSet`
|
||||
|
||||
**Actions**:
|
||||
- `list()`: List keywords with filtering
|
||||
- `create()`: Create keyword
|
||||
- `retrieve()`: Get keyword details
|
||||
- `update()`: Update keyword
|
||||
- `destroy()`: Delete keyword
|
||||
- `auto_cluster()`: Auto-cluster keywords using AI
|
||||
- `bulk_delete()`: Bulk delete keywords
|
||||
- `bulk_update_status()`: Bulk update keyword status
|
||||
- `export_csv()`: Export keywords to CSV
|
||||
- `import_csv()`: Import keywords from CSV
|
||||
|
||||
**Filtering**:
|
||||
- Search: `keyword` field
|
||||
- Filters: `status`, `cluster_id`, `intent`
|
||||
- Custom: `difficulty_min`, `difficulty_max`, `volume_min`, `volume_max`
|
||||
- Ordering: `created_at`, `volume`, `difficulty`
|
||||
|
||||
#### ClusterViewSet
|
||||
**File**: `modules/planner/views.py`
|
||||
|
||||
**Inherits**: `SiteSectorModelViewSet`
|
||||
|
||||
**Actions**:
|
||||
- `list()`: List clusters
|
||||
- `create()`: Create cluster
|
||||
- `retrieve()`: Get cluster details
|
||||
- `update()`: Update cluster
|
||||
- `destroy()`: Delete cluster
|
||||
- `auto_generate_ideas()`: Auto-generate content ideas for clusters
|
||||
|
||||
#### ContentIdeasViewSet
|
||||
**File**: `modules/planner/views.py`
|
||||
|
||||
**Inherits**: `SiteSectorModelViewSet`
|
||||
|
||||
**Actions**:
|
||||
- `list()`: List content ideas
|
||||
- `create()`: Create content idea
|
||||
- `retrieve()`: Get content idea details
|
||||
- `update()`: Update content idea
|
||||
- `destroy()`: Delete content idea
|
||||
|
||||
### Writer ViewSets
|
||||
|
||||
#### TasksViewSet
|
||||
**File**: `modules/writer/views.py`
|
||||
|
||||
**Inherits**: `SiteSectorModelViewSet`
|
||||
|
||||
**Actions**:
|
||||
- `list()`: List tasks
|
||||
- `create()`: Create task
|
||||
- `retrieve()`: Get task details
|
||||
- `update()`: Update task
|
||||
- `destroy()`: Delete task
|
||||
- `auto_generate_content()`: Auto-generate content for tasks
|
||||
- `auto_generate_images()`: Auto-generate images for tasks
|
||||
- `bulk_delete()`: Bulk delete tasks
|
||||
- `bulk_update()`: Bulk update task status
|
||||
|
||||
**Filtering**:
|
||||
- Search: `title`, `keywords`
|
||||
- Filters: `status`, `cluster_id`, `content_type`, `content_structure`
|
||||
- Ordering: `title`, `created_at`, `word_count`, `status`
|
||||
|
||||
### System ViewSets
|
||||
|
||||
#### IntegrationSettingsViewSet
|
||||
**File**: `modules/system/integration_views.py`
|
||||
|
||||
**Inherits**: `viewsets.ViewSet`
|
||||
|
||||
**Actions**:
|
||||
- `list()`: List integrations
|
||||
- `retrieve()`: Get integration settings
|
||||
- `update()`: Save integration settings
|
||||
- `save_post()`: Save integration settings (POST)
|
||||
- `test_connection()`: Test API connection
|
||||
- `test_openai()`: Test OpenAI connection
|
||||
- `test_runware()`: Test Runware connection
|
||||
- `generate_image()`: Test image generation
|
||||
- `task_progress()`: Get Celery task progress
|
||||
|
||||
#### AIPromptViewSet
|
||||
**File**: `modules/system/views.py`
|
||||
|
||||
**Inherits**: `AccountModelViewSet`
|
||||
|
||||
**Actions**:
|
||||
- `list()`: List prompts
|
||||
- `create()`: Create prompt
|
||||
- `retrieve()`: Get prompt details
|
||||
- `update()`: Update prompt
|
||||
- `destroy()`: Delete prompt
|
||||
- `reset_to_default()`: Reset prompt to default value
|
||||
|
||||
#### AuthorProfileViewSet
|
||||
**File**: `modules/system/views.py`
|
||||
|
||||
**Inherits**: `AccountModelViewSet`
|
||||
|
||||
**Actions**:
|
||||
- `list()`: List author profiles
|
||||
- `create()`: Create author profile
|
||||
- `retrieve()`: Get author profile details
|
||||
- `update()`: Update author profile
|
||||
- `destroy()`: Delete author profile
|
||||
|
||||
---
|
||||
|
||||
## Serializers
|
||||
|
||||
### Planner Serializers
|
||||
|
||||
#### KeywordSerializer
|
||||
**File**: `modules/planner/serializers.py`
|
||||
|
||||
**Fields**: All Keyword model fields
|
||||
|
||||
**Validation**: Validates keyword uniqueness, cluster belongs to same sector
|
||||
|
||||
#### ClusterSerializer
|
||||
**File**: `modules/planner/cluster_serializers.py`
|
||||
|
||||
**Fields**: All Cluster model fields
|
||||
|
||||
**Read-Only Fields**: `keywords_count`, `volume` (calculated)
|
||||
|
||||
#### ContentIdeasSerializer
|
||||
**File**: `modules/planner/serializers.py`
|
||||
|
||||
**Fields**: All ContentIdeas model fields
|
||||
|
||||
### Writer Serializers
|
||||
|
||||
#### TasksSerializer
|
||||
**File**: `modules/writer/serializers.py`
|
||||
|
||||
**Fields**: All Tasks model fields
|
||||
|
||||
#### ContentSerializer
|
||||
**File**: `modules/writer/serializers.py`
|
||||
|
||||
**Fields**: All Content model fields
|
||||
|
||||
#### ImagesSerializer
|
||||
**File**: `modules/writer/serializers.py`
|
||||
|
||||
**Fields**: All Images model fields
|
||||
|
||||
### System Serializers
|
||||
|
||||
#### AIPromptSerializer
|
||||
**File**: `modules/system/serializers.py`
|
||||
|
||||
**Fields**: All AIPrompt model fields
|
||||
|
||||
#### IntegrationSettingsSerializer
|
||||
**File**: `modules/system/serializers.py`
|
||||
|
||||
**Fields**: All IntegrationSettings model fields
|
||||
|
||||
---
|
||||
|
||||
## Celery Tasks
|
||||
|
||||
### Planner Tasks
|
||||
|
||||
#### auto_cluster_keywords_task
|
||||
**File**: `modules/planner/tasks.py`
|
||||
|
||||
**Purpose**: Auto-cluster keywords using AI.
|
||||
|
||||
**Parameters**:
|
||||
- `keyword_ids`: List of keyword IDs
|
||||
- `account_id`: Account ID
|
||||
- `site_id`: Site ID
|
||||
- `sector_id`: Sector ID
|
||||
|
||||
**Progress Tracking**: Updates progress with request_steps and response_steps
|
||||
|
||||
**Calls**: `_auto_cluster_keywords_core()`
|
||||
|
||||
#### auto_generate_ideas_task
|
||||
**File**: `modules/planner/tasks.py`
|
||||
|
||||
**Purpose**: Auto-generate content ideas for clusters.
|
||||
|
||||
**Parameters**:
|
||||
- `cluster_ids`: List of cluster IDs
|
||||
- `account_id`: Account ID
|
||||
|
||||
**Progress Tracking**: Updates progress for each cluster
|
||||
|
||||
**Calls**: `_generate_single_idea_core()` for each cluster
|
||||
|
||||
### Writer Tasks
|
||||
|
||||
#### auto_generate_content_task
|
||||
**File**: `modules/writer/tasks.py`
|
||||
|
||||
**Purpose**: Auto-generate content for tasks.
|
||||
|
||||
**Parameters**:
|
||||
- `task_ids`: List of task IDs
|
||||
- `account_id`: Account ID
|
||||
|
||||
**Progress Tracking**: Updates progress for each task
|
||||
|
||||
**Calls**: `AIProcessor.generate_content()`
|
||||
|
||||
#### auto_generate_images_task
|
||||
**File**: `modules/writer/tasks.py`
|
||||
|
||||
**Purpose**: Auto-generate images for tasks.
|
||||
|
||||
**Parameters**:
|
||||
- `task_ids`: List of task IDs
|
||||
- `account_id`: Account ID
|
||||
|
||||
**Progress Tracking**: Updates progress for each task
|
||||
|
||||
**Calls**: `AIProcessor.extract_image_prompts()` and `AIProcessor.generate_image()`
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Base URL
|
||||
|
||||
`/api/v1/`
|
||||
|
||||
### Planner Endpoints
|
||||
|
||||
- `GET /api/v1/planner/keywords/` - List keywords
|
||||
- `POST /api/v1/planner/keywords/` - Create keyword
|
||||
- `GET /api/v1/planner/keywords/{id}/` - Get keyword
|
||||
- `PUT /api/v1/planner/keywords/{id}/` - Update keyword
|
||||
- `DELETE /api/v1/planner/keywords/{id}/` - Delete keyword
|
||||
- `POST /api/v1/planner/keywords/auto_cluster/` - Auto-cluster keywords
|
||||
- `POST /api/v1/planner/keywords/bulk_delete/` - Bulk delete keywords
|
||||
- `POST /api/v1/planner/keywords/bulk_update_status/` - Bulk update status
|
||||
- `GET /api/v1/planner/keywords/export_csv/` - Export keywords
|
||||
- `POST /api/v1/planner/keywords/import_csv/` - Import keywords
|
||||
- `GET /api/v1/planner/clusters/` - List clusters
|
||||
- `POST /api/v1/planner/clusters/auto_generate_ideas/` - Auto-generate ideas
|
||||
- `GET /api/v1/planner/ideas/` - List content ideas
|
||||
|
||||
### Writer Endpoints
|
||||
|
||||
- `GET /api/v1/writer/tasks/` - List tasks
|
||||
- `POST /api/v1/writer/tasks/auto_generate_content/` - Auto-generate content
|
||||
- `POST /api/v1/writer/tasks/auto_generate_images/` - Auto-generate images
|
||||
|
||||
### System Endpoints
|
||||
|
||||
- `GET /api/v1/system/settings/integrations/{pk}/` - Get integration settings
|
||||
- `PUT /api/v1/system/settings/integrations/{pk}/` - Save integration settings
|
||||
- `POST /api/v1/system/settings/integrations/{pk}/test_openai/` - Test OpenAI
|
||||
- `POST /api/v1/system/settings/integrations/{pk}/test_runware/` - Test Runware
|
||||
- `POST /api/v1/system/settings/integrations/{pk}/generate_image/` - Test image generation
|
||||
- `GET /api/v1/system/settings/task_progress/{task_id}/` - Get task progress
|
||||
|
||||
---
|
||||
|
||||
## Base Classes
|
||||
|
||||
### AccountModelViewSet
|
||||
|
||||
**File**: `api/base.py`
|
||||
|
||||
**Purpose**: Base ViewSet with automatic account filtering.
|
||||
|
||||
**Features**:
|
||||
- Automatic account filtering
|
||||
- Admin/Developer override
|
||||
- Account context in serializers
|
||||
|
||||
### SiteSectorModelViewSet
|
||||
|
||||
**File**: `api/base.py`
|
||||
|
||||
**Purpose**: Base ViewSet with site/sector filtering.
|
||||
|
||||
**Features**:
|
||||
- Account filtering (inherited)
|
||||
- Site access control
|
||||
- Sector validation
|
||||
- Accessible sites/sectors in serializer context
|
||||
|
||||
---
|
||||
|
||||
## Middleware
|
||||
|
||||
### AccountContextMiddleware
|
||||
|
||||
**File**: `middleware/account.py`
|
||||
|
||||
**Purpose**: Sets `request.account` from JWT token.
|
||||
|
||||
**Functionality**:
|
||||
- Extracts account ID from JWT token
|
||||
- Loads Account object
|
||||
- Sets `request.account`
|
||||
|
||||
### ResourceTrackerMiddleware
|
||||
|
||||
**File**: `middleware/resource_tracker.py`
|
||||
|
||||
**Purpose**: Tracks API request metrics.
|
||||
|
||||
**Functionality**:
|
||||
- Tracks CPU, memory, I/O usage
|
||||
- Stores metrics in cache
|
||||
- Provides metrics endpoint
|
||||
|
||||
---
|
||||
|
||||
## Utilities
|
||||
|
||||
### AIProcessor
|
||||
|
||||
**File**: `utils/ai_processor.py`
|
||||
|
||||
**Purpose**: Unified AI interface for all AI operations.
|
||||
|
||||
**Methods**:
|
||||
- `cluster_keywords()`: Cluster keywords using AI
|
||||
- `generate_ideas()`: Generate content ideas
|
||||
- `generate_content()`: Generate text content
|
||||
- `extract_image_prompts()`: Extract image prompts from content
|
||||
- `generate_image()`: Generate images using OpenAI DALL-E or Runware
|
||||
|
||||
**See**: AI Functions documentation for complete details
|
||||
|
||||
### Content Normalizer
|
||||
|
||||
**File**: `utils/content_normalizer.py`
|
||||
|
||||
**Purpose**: Content processing utilities.
|
||||
|
||||
**Functions**:
|
||||
- `_extract_body_content()`: Extract body content from HTML
|
||||
|
||||
---
|
||||
|
||||
## Modules
|
||||
|
||||
### Planner Module
|
||||
|
||||
**Purpose**: Keyword management and content planning.
|
||||
|
||||
**Models**: Keywords, Clusters, ContentIdeas
|
||||
|
||||
**ViewSets**: KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
|
||||
|
||||
**Tasks**: auto_cluster_keywords_task, auto_generate_ideas_task
|
||||
|
||||
### Writer Module
|
||||
|
||||
**Purpose**: Content generation and management.
|
||||
|
||||
**Models**: Tasks, Content, Images
|
||||
|
||||
**ViewSets**: TasksViewSet
|
||||
|
||||
**Tasks**: auto_generate_content_task, auto_generate_images_task
|
||||
|
||||
### System Module
|
||||
|
||||
**Purpose**: System settings, prompts, and integrations.
|
||||
|
||||
**Models**: AIPrompt, IntegrationSettings, AuthorProfile, Strategy
|
||||
|
||||
**ViewSets**: AIPromptViewSet, AuthorProfileViewSet, IntegrationSettingsViewSet
|
||||
|
||||
**Utilities**: Default prompts, prompt loading
|
||||
|
||||
### Billing Module
|
||||
|
||||
**Purpose**: Credits, transactions, and usage tracking.
|
||||
|
||||
**Models**: CreditTransaction, UsageLog
|
||||
|
||||
**ViewSets**: CreditTransactionViewSet, UsageLogViewSet
|
||||
|
||||
**Services**: CreditService
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
The IGNY8 backend is built on:
|
||||
|
||||
1. **Django + DRF**: Robust web framework with RESTful API
|
||||
2. **Multi-Tenancy**: Complete account isolation with automatic filtering
|
||||
3. **Modular Architecture**: Clear module boundaries with shared utilities
|
||||
4. **Celery Integration**: Asynchronous task processing for long-running operations
|
||||
5. **Base ViewSets**: Consistent access control and filtering
|
||||
6. **AI Integration**: Unified AIProcessor for all AI operations
|
||||
7. **Progress Tracking**: Real-time progress updates for Celery tasks
|
||||
8. **Account/Site/Sector Hierarchy**: Hierarchical data organization
|
||||
|
||||
This architecture ensures scalability, maintainability, and extensibility while providing a robust API for the frontend.
|
||||
|
||||
739
docs/05-ACCOUNT-USER-PLAN.md
Normal file
739
docs/05-ACCOUNT-USER-PLAN.md
Normal file
@@ -0,0 +1,739 @@
|
||||
# IGNY8 Account, User, Plan, Sites, Credits Documentation
|
||||
|
||||
**Version:** 1.0
|
||||
**Last Updated:** 2025-01-XX
|
||||
**Purpose:** Complete documentation of multi-tenancy architecture, access control, plans, credits, and related systems.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Overview](#overview)
|
||||
2. [Account Model](#account-model)
|
||||
3. [User Model](#user-model)
|
||||
4. [Plan Model](#plan-model)
|
||||
5. [Subscription Model](#subscription-model)
|
||||
6. [Site Model](#site-model)
|
||||
7. [Sector Model](#sector-model)
|
||||
8. [Industry & IndustrySector Models](#industry--industrysector-models)
|
||||
9. [Access Control](#access-control)
|
||||
10. [Credits System](#credits-system)
|
||||
11. [Plan Limits](#plan-limits)
|
||||
12. [Multi-Tenancy Architecture](#multi-tenancy-architecture)
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The IGNY8 platform uses a multi-account architecture with the following core entities:
|
||||
|
||||
- **Account**: Top-level organization/workspace
|
||||
- **User**: Individual user accounts with roles
|
||||
- **Plan**: Subscription plan templates with limits
|
||||
- **Subscription**: Active subscription linking Account to Plan
|
||||
- **Site**: Workspace within an Account (1-N relationship)
|
||||
- **Sector**: Content category within a Site (1-5 per site)
|
||||
- **Industry**: Global industry templates
|
||||
- **IndustrySector**: Industry sector templates
|
||||
|
||||
### Key Relationships
|
||||
|
||||
```
|
||||
Account (1) ──< (N) User
|
||||
Account (1) ──< (1) Subscription ──> (1) Plan
|
||||
Account (1) ──< (N) Site
|
||||
Site (1) ──< (1-5) Sector
|
||||
Industry (1) ──< (N) IndustrySector
|
||||
Site (1) ──> (1) Industry (optional)
|
||||
Sector (1) ──> (1) IndustrySector (optional, template reference)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Account Model
|
||||
|
||||
**File**: `backend/igny8_core/auth/models.py`
|
||||
|
||||
**Table**: `igny8_accounts`
|
||||
|
||||
### Fields
|
||||
|
||||
- `id`: Primary key
|
||||
- `name`: CharField - Account name
|
||||
- `slug`: SlugField - Unique slug identifier
|
||||
- `owner`: ForeignKey to User - Account owner
|
||||
- `stripe_customer_id`: CharField - Stripe customer ID (optional)
|
||||
- `plan`: ForeignKey to Plan - Current subscription plan
|
||||
- `credits`: IntegerField - Current credit balance (default: 0)
|
||||
- `status`: CharField - Account status (choices: active, suspended, trial, cancelled)
|
||||
- `created_at`: DateTimeField - Creation timestamp
|
||||
- `updated_at`: DateTimeField - Last update timestamp
|
||||
|
||||
### Status Values
|
||||
|
||||
- `active`: Normal operation
|
||||
- `suspended`: Access temporarily revoked
|
||||
- `trial`: Limited trial access
|
||||
- `cancelled`: Account terminated
|
||||
|
||||
### Methods
|
||||
|
||||
#### is_system_account()
|
||||
**Returns**: `True` if account is a system account (aws-admin, default-account, default)
|
||||
|
||||
**Purpose**: System accounts bypass all filtering restrictions
|
||||
|
||||
|
||||
### Relationships
|
||||
|
||||
- **One-to-Many**: Users (via `user.account`)
|
||||
- **One-to-One**: Subscription (via `account.subscription`)
|
||||
- **One-to-Many**: Sites (via `site.account`)
|
||||
|
||||
|
||||
---
|
||||
|
||||
## User Model
|
||||
|
||||
**File**: `backend/igny8_core/auth/models.py`
|
||||
|
||||
**Table**: `igny8_users`
|
||||
|
||||
**Inherits**: `AbstractUser` (Django's built-in user model)
|
||||
|
||||
### Fields
|
||||
|
||||
- `id`: Primary key (inherited from AbstractUser)
|
||||
- `email`: EmailField - Unique, USERNAME_FIELD
|
||||
- `username`: CharField - Username
|
||||
- `password`: Hashed password (inherited)
|
||||
- `account`: ForeignKey to Account - User's account (null=True, blank=True)
|
||||
- `role`: CharField - User role (choices: developer, owner, admin, editor, viewer, system_bot)
|
||||
- `created_at`: DateTimeField - Creation timestamp
|
||||
- `updated_at`: DateTimeField - Last update timestamp
|
||||
|
||||
### Role Hierarchy (from highest to lowest)
|
||||
|
||||
1. **developer**: Full system access, bypasses all restrictions
|
||||
2. **owner**: Full access to account, can manage users and billing
|
||||
3. **admin**: Admin access to account, can manage content and users
|
||||
4. **editor**: Can edit content, manage clusters/tasks
|
||||
5. **viewer**: Read-only access
|
||||
6. **system_bot**: System automation user
|
||||
|
||||
### Methods
|
||||
|
||||
#### has_role(*roles)
|
||||
**Parameters**: `*roles` - Variable number of role strings
|
||||
|
||||
**Returns**: `True` if user has any of the specified roles
|
||||
|
||||
#### is_owner_or_admin()
|
||||
**Returns**: `True` if role is owner or admin
|
||||
|
||||
#### is_developer()
|
||||
**Returns**: `True` if role is developer or is_superuser
|
||||
|
||||
#### is_admin_or_developer()
|
||||
**Returns**: `True` if role is admin or developer (bypasses restrictions)
|
||||
|
||||
**Purpose**: Admin/Developer users bypass account/site/sector filtering
|
||||
|
||||
#### is_system_account_user()
|
||||
**Returns**: `True` if user belongs to system account
|
||||
|
||||
**Purpose**: System account users bypass all filtering restrictions
|
||||
|
||||
#### get_accessible_sites()
|
||||
**Returns**: QuerySet of sites user can access
|
||||
|
||||
**Access Control Logic**:
|
||||
- **System Account Users**: All active sites across all accounts
|
||||
- **Developers**: All active sites across all accounts
|
||||
- **Owners/Admins**: All sites in their account
|
||||
- **Editors/Viewers**: Only sites explicitly granted via `SiteUserAccess`
|
||||
|
||||
|
||||
### Relationships
|
||||
|
||||
- **Many-to-One**: Account (via `user.account`)
|
||||
- **Many-to-Many**: Sites (via `SiteUserAccess`)
|
||||
|
||||
---
|
||||
|
||||
## Plan Model
|
||||
|
||||
**File**: `backend/igny8_core/auth/models.py`
|
||||
|
||||
**Table**: `igny8_plans`
|
||||
|
||||
### Fields
|
||||
|
||||
#### Plan Information
|
||||
- `id`: Primary key
|
||||
- `name`: CharField - Plan name (e.g., "Free", "Starter", "Growth")
|
||||
- `slug`: SlugField - Unique slug identifier
|
||||
- `price`: DecimalField - Monthly price
|
||||
- `billing_cycle`: CharField - Billing cycle (choices: monthly, annual)
|
||||
- `features`: JSONField - Array of feature strings (e.g., ['ai_writer', 'image_gen', 'auto_publish'])
|
||||
- `is_active`: BooleanField - Whether plan is available for subscription
|
||||
|
||||
#### User/Site/Scope Limits
|
||||
- `max_users`: IntegerField - Total users allowed per account (default: 1, min: 1)
|
||||
- `max_sites`: IntegerField - Maximum sites allowed (default: 1, min: 1)
|
||||
- `max_industries`: IntegerField - Optional limit for industries/sectors (nullable)
|
||||
- `max_author_profiles`: IntegerField - Limit for saved writing styles (default: 5)
|
||||
|
||||
#### Planner Limits
|
||||
- `max_keywords`: IntegerField - Total keywords allowed globally (default: 1000)
|
||||
- `max_clusters`: IntegerField - Total clusters allowed globally (default: 100)
|
||||
- `max_content_ideas`: IntegerField - Total content ideas allowed globally (default: 300)
|
||||
- `daily_cluster_limit`: IntegerField - Max clusters per day (default: 10)
|
||||
- `daily_keyword_import_limit`: IntegerField - Seed keywords import limit per day (default: 100)
|
||||
- `monthly_cluster_ai_credits`: IntegerField - AI credits for clustering (default: 50)
|
||||
|
||||
#### Writer Limits
|
||||
- `daily_content_tasks`: IntegerField - Max content tasks per day (default: 10)
|
||||
- `daily_ai_requests`: IntegerField - Total AI executions per day (default: 50)
|
||||
- `monthly_word_count_limit`: IntegerField - Monthly word limit for generated content (default: 50000)
|
||||
- `monthly_content_ai_credits`: IntegerField - AI credit pool for content generation (default: 200)
|
||||
|
||||
#### Image Generation Limits
|
||||
- `monthly_image_count`: IntegerField - Max images per month (default: 100)
|
||||
- `daily_image_generation_limit`: IntegerField - Max images per day (default: 25)
|
||||
- `monthly_image_ai_credits`: IntegerField - AI credit pool for images (default: 100)
|
||||
- `max_images_per_task`: IntegerField - Max images per content task (default: 4, min: 1)
|
||||
- `image_model_choices`: JSONField - Allowed image models (e.g., ['dalle3', 'hidream'])
|
||||
|
||||
#### AI Request Controls
|
||||
- `daily_ai_request_limit`: IntegerField - Global daily AI request cap (default: 100)
|
||||
- `monthly_ai_credit_limit`: IntegerField - Unified credit ceiling per month (default: 500)
|
||||
|
||||
#### Billing & Credits
|
||||
- `included_credits`: IntegerField - Monthly credits included (default: 0)
|
||||
- `extra_credit_price`: DecimalField - Price per additional credit (default: 0.01)
|
||||
- `allow_credit_topup`: BooleanField - Can user purchase more credits? (default: True)
|
||||
- `auto_credit_topup_threshold`: IntegerField - Auto top-up trigger point (nullable)
|
||||
- `auto_credit_topup_amount`: IntegerField - Credits to auto-buy (nullable)
|
||||
|
||||
#### Stripe Integration
|
||||
- `stripe_product_id`: CharField - Stripe product ID (nullable)
|
||||
- `stripe_price_id`: CharField - Stripe price ID (nullable)
|
||||
|
||||
#### Legacy Fields
|
||||
- `credits_per_month`: IntegerField - DEPRECATED: Use included_credits instead (default: 0)
|
||||
|
||||
### Methods
|
||||
|
||||
#### clean()
|
||||
**Purpose**: Validates plan limits
|
||||
|
||||
**Validations**:
|
||||
- `max_sites` must be >= 1
|
||||
- `included_credits` must be >= 0
|
||||
|
||||
#### get_effective_credits_per_month()
|
||||
**Returns**: `included_credits` if set, otherwise `credits_per_month` (backward compatibility)
|
||||
|
||||
### Relationships
|
||||
|
||||
- **One-to-Many**: Accounts (via `account.plan`)
|
||||
|
||||
### Plan Limits Enforcement
|
||||
|
||||
**Current Status**: Plan limits are defined but **not currently enforced** in AI functions.
|
||||
|
||||
**Future Implementation**: Plan limits should be checked before allowing operations.
|
||||
|
||||
---
|
||||
|
||||
## Subscription Model
|
||||
|
||||
**File**: `backend/igny8_core/auth/models.py`
|
||||
|
||||
**Table**: `igny8_subscriptions`
|
||||
|
||||
### Fields
|
||||
|
||||
- `id`: Primary key
|
||||
- `account`: OneToOneField to Account - Account subscription
|
||||
- `stripe_subscription_id`: CharField - Unique Stripe subscription ID
|
||||
- `status`: CharField - Subscription status (choices: active, past_due, canceled, trialing)
|
||||
- `current_period_start`: DateTimeField - Current billing period start
|
||||
- `current_period_end`: DateTimeField - Current billing period end
|
||||
- `cancel_at_period_end`: BooleanField - Cancel at period end flag (default: False)
|
||||
- `created_at`: DateTimeField - Creation timestamp
|
||||
- `updated_at`: DateTimeField - Last update timestamp
|
||||
|
||||
### Status Values
|
||||
|
||||
- `active`: Subscription is active and paid
|
||||
- `past_due`: Payment failed, subscription past due
|
||||
- `canceled`: Subscription canceled
|
||||
- `trialing`: In trial period
|
||||
|
||||
### Methods
|
||||
|
||||
|
||||
### Relationships
|
||||
|
||||
- **One-to-One**: Account (via `account.subscription`)
|
||||
|
||||
---
|
||||
|
||||
## Site Model
|
||||
|
||||
**File**: `backend/igny8_core/auth/models.py`
|
||||
|
||||
**Table**: `igny8_sites`
|
||||
|
||||
**Inherits**: `AccountBaseModel`
|
||||
|
||||
### Fields
|
||||
|
||||
- `id`: Primary key
|
||||
- `account`: ForeignKey to Account - Account this site belongs to
|
||||
- `name`: CharField - Site name
|
||||
- `slug`: SlugField - Unique slug per account
|
||||
- `domain`: URLField - Primary domain URL (optional)
|
||||
- `description`: TextField - Site description (optional)
|
||||
- `industry`: ForeignKey to Industry - Industry this site belongs to (optional)
|
||||
- `is_active`: BooleanField - Whether site is active (default: True)
|
||||
- `status`: CharField - Site status (choices: active, inactive, suspended, default: active)
|
||||
- `wp_url`: URLField - WordPress site URL (optional, for WordPress integration)
|
||||
- `wp_username`: CharField - WordPress username (optional)
|
||||
- `wp_app_password`: CharField - WordPress app password (optional)
|
||||
- `created_at`: DateTimeField - Creation timestamp
|
||||
- `updated_at`: DateTimeField - Last update timestamp
|
||||
|
||||
### Unique Constraint
|
||||
|
||||
- `(account, slug)` - Slug unique per account
|
||||
|
||||
### Methods
|
||||
|
||||
#### get_active_sectors_count()
|
||||
**Returns**: Count of active sectors for this site
|
||||
|
||||
#### can_add_sector()
|
||||
**Returns**: `True` if site can add another sector (max 5 sectors per site)
|
||||
|
||||
### Relationships
|
||||
|
||||
- **Many-to-One**: Account (via `site.account`)
|
||||
- **Many-to-One**: Industry (via `site.industry`, optional)
|
||||
- **One-to-Many**: Sectors (via `sector.site`)
|
||||
- **Many-to-Many**: Users (via `SiteUserAccess`)
|
||||
|
||||
### Site Activation
|
||||
|
||||
**Important**: Multiple sites can be active simultaneously. The previous restriction of "only 1 site can be active at one time" has been removed.
|
||||
|
||||
---
|
||||
|
||||
## Sector Model
|
||||
|
||||
**File**: `backend/igny8_core/auth/models.py`
|
||||
|
||||
**Table**: `igny8_sectors`
|
||||
|
||||
**Inherits**: `AccountBaseModel`
|
||||
|
||||
### Fields
|
||||
|
||||
- `id`: Primary key
|
||||
- `account`: ForeignKey to Account - Account this sector belongs to
|
||||
- `site`: ForeignKey to Site - Site this sector belongs to
|
||||
- `industry_sector`: ForeignKey to IndustrySector - Reference to industry sector template (optional)
|
||||
- `name`: CharField - Sector name
|
||||
- `slug`: SlugField - Unique slug per site
|
||||
- `description`: TextField - Sector description (optional)
|
||||
- `is_active`: BooleanField - Whether sector is active (default: True)
|
||||
- `status`: CharField - Sector status (choices: active, inactive, default: active)
|
||||
- `created_at`: DateTimeField - Creation timestamp
|
||||
- `updated_at`: DateTimeField - Last update timestamp
|
||||
|
||||
### Unique Constraint
|
||||
|
||||
- `(site, slug)` - Slug unique per site
|
||||
|
||||
### Validation
|
||||
|
||||
**On Save**:
|
||||
1. Automatically sets `account` from `site.account`
|
||||
2. Validates that `industry_sector.industry` matches `site.industry` (if both set)
|
||||
3. Validates sector limit: Maximum 5 active sectors per site
|
||||
|
||||
### Methods
|
||||
|
||||
#### industry (property)
|
||||
**Returns**: Industry for this sector (from `industry_sector.industry` if set)
|
||||
|
||||
### Relationships
|
||||
|
||||
- **Many-to-One**: Account (via `sector.account`)
|
||||
- **Many-to-One**: Site (via `sector.site`)
|
||||
- **Many-to-One**: IndustrySector (via `sector.industry_sector`, optional, template reference)
|
||||
- **One-to-Many**: Keywords, Clusters, ContentIdeas, Tasks (via their `sector` field)
|
||||
|
||||
---
|
||||
|
||||
## Industry & IndustrySector Models
|
||||
|
||||
### Industry Model
|
||||
|
||||
**File**: `backend/igny8_core/auth/models.py`
|
||||
|
||||
**Table**: `igny8_industries`
|
||||
|
||||
**Purpose**: Global industry templates.
|
||||
|
||||
### Fields
|
||||
|
||||
- `id`: Primary key
|
||||
- `name`: CharField - Industry name (unique)
|
||||
- `slug`: SlugField - Unique slug identifier
|
||||
- `description`: TextField - Industry description (optional)
|
||||
- `is_active`: BooleanField - Whether industry is active (default: True)
|
||||
- `created_at`: DateTimeField - Creation timestamp
|
||||
- `updated_at`: DateTimeField - Last update timestamp
|
||||
|
||||
### Relationships
|
||||
|
||||
- **One-to-Many**: Sites (via `site.industry`)
|
||||
- **One-to-Many**: IndustrySectors (via `industry_sector.industry`)
|
||||
|
||||
### IndustrySector Model
|
||||
|
||||
**File**: `backend/igny8_core/auth/models.py`
|
||||
|
||||
**Table**: `igny8_industry_sectors`
|
||||
|
||||
**Purpose**: Sector templates within industries.
|
||||
|
||||
### Fields
|
||||
|
||||
- `id`: Primary key
|
||||
- `industry`: ForeignKey to Industry - Industry this sector belongs to
|
||||
- `name`: CharField - Sector name
|
||||
- `slug`: SlugField - Unique slug per industry
|
||||
- `description`: TextField - Sector description (optional)
|
||||
- `suggested_keywords`: JSONField - List of suggested keywords for this sector template
|
||||
- `is_active`: BooleanField - Whether sector is active (default: True)
|
||||
- `created_at`: DateTimeField - Creation timestamp
|
||||
- `updated_at`: DateTimeField - Last update timestamp
|
||||
|
||||
### Unique Constraint
|
||||
|
||||
- `(industry, slug)` - Slug unique per industry
|
||||
|
||||
### Relationships
|
||||
|
||||
- **Many-to-One**: Industry (via `industry_sector.industry`)
|
||||
- **One-to-Many**: Sectors (via `sector.industry_sector`, template reference)
|
||||
- **One-to-Many**: SeedKeywords (via `seed_keyword.sector`)
|
||||
|
||||
---
|
||||
|
||||
## Access Control
|
||||
|
||||
### Account Isolation
|
||||
|
||||
**Principle**: All data is isolated by account.
|
||||
|
||||
**Implementation**:
|
||||
- All models inherit `AccountBaseModel` (has `account` ForeignKey)
|
||||
- All ViewSets inherit `AccountModelViewSet` (filters by `request.account`)
|
||||
- Middleware sets `request.account` from JWT token
|
||||
|
||||
**Access Control**:
|
||||
- **Admin/Developer users**: Bypass account filtering (see all accounts)
|
||||
- **System account users**: Bypass account filtering (see all accounts)
|
||||
- **Regular users**: Only see data from their account
|
||||
|
||||
### Site Access Control
|
||||
|
||||
**Principle**: Users can only access sites they have permission to access.
|
||||
|
||||
**Implementation**:
|
||||
- `User.get_accessible_sites()` returns sites user can access
|
||||
- ViewSets filter by accessible sites
|
||||
- `SiteUserAccess` model controls explicit site access for editors/viewers
|
||||
|
||||
**Access Levels**:
|
||||
- **System Account Users**: All active sites across all accounts
|
||||
- **Developers**: All active sites across all accounts
|
||||
- **Owners/Admins**: All sites in their account
|
||||
- **Editors/Viewers**: Only sites explicitly granted via `SiteUserAccess`
|
||||
|
||||
### SiteUserAccess Model
|
||||
|
||||
**File**: `backend/igny8_core/auth/models.py`
|
||||
|
||||
**Table**: `igny8_site_user_access`
|
||||
|
||||
**Purpose**: Many-to-many relationship between Users and Sites for explicit access control.
|
||||
|
||||
### Fields
|
||||
|
||||
- `id`: Primary key
|
||||
- `user`: ForeignKey to User
|
||||
- `site`: ForeignKey to Site
|
||||
- `granted_at`: DateTimeField - When access was granted
|
||||
- `granted_by`: ForeignKey to User - Who granted access (optional)
|
||||
|
||||
### Unique Constraint
|
||||
|
||||
- `(user, site)` - One access record per user-site pair
|
||||
|
||||
### Usage
|
||||
|
||||
- Owners and Admins have automatic access to all sites in their account
|
||||
- Editors and Viewers require explicit `SiteUserAccess` records
|
||||
- System account users and Developers bypass this check
|
||||
|
||||
### Role-Based Access Control (RBAC)
|
||||
|
||||
**Role Hierarchy**:
|
||||
1. **developer**: Full system access, bypasses all restrictions
|
||||
2. **owner**: Full account access, can manage users and billing
|
||||
3. **admin**: Account admin access, can manage content and users
|
||||
4. **editor**: Content editing access, can manage clusters/tasks
|
||||
5. **viewer**: Read-only access
|
||||
|
||||
**Permission Checks**:
|
||||
- `user.has_role(*roles)`: Check if user has any of the specified roles
|
||||
- `user.is_owner_or_admin()`: Check if user is owner or admin
|
||||
- `user.is_developer()`: Check if user is developer
|
||||
- `user.is_admin_or_developer()`: Check if user is admin or developer (bypasses restrictions)
|
||||
|
||||
---
|
||||
|
||||
## Credits System
|
||||
|
||||
### Account Credits
|
||||
|
||||
**Field**: `Account.credits` (IntegerField, default: 0)
|
||||
|
||||
**Purpose**: Current credit balance for the account.
|
||||
|
||||
**Usage**: Credits are deducted for AI operations and added through purchases or subscriptions.
|
||||
|
||||
### Credit Models
|
||||
|
||||
#### CreditTransaction
|
||||
|
||||
**File**: `backend/igny8_core/modules/billing/models.py`
|
||||
|
||||
**Table**: `igny8_credit_transactions`
|
||||
|
||||
**Purpose**: Track all credit transactions (additions, deductions).
|
||||
|
||||
### Fields
|
||||
|
||||
- `id`: Primary key
|
||||
- `account`: ForeignKey to Account
|
||||
- `transaction_type`: CharField - Type of transaction (choices: purchase, subscription, refund, deduction, adjustment)
|
||||
- `amount`: IntegerField - Positive for additions, negative for deductions
|
||||
- `balance_after`: IntegerField - Credit balance after this transaction
|
||||
- `description`: CharField - Transaction description
|
||||
- `metadata`: JSONField - Additional context (AI call details, etc.)
|
||||
- `created_at`: DateTimeField - Creation timestamp
|
||||
|
||||
#### CreditUsageLog
|
||||
|
||||
**File**: `backend/igny8_core/modules/billing/models.py`
|
||||
|
||||
**Table**: `igny8_credit_usage_logs`
|
||||
|
||||
**Purpose**: Detailed log of credit usage per AI operation.
|
||||
|
||||
### Fields
|
||||
|
||||
- `id`: Primary key
|
||||
- `account`: ForeignKey to Account
|
||||
- `operation_type`: CharField - Type of operation (choices: clustering, ideas, content, images, reparse)
|
||||
- `credits_used`: IntegerField - Number of credits used
|
||||
- `cost_usd`: DecimalField - Cost in USD (optional)
|
||||
- `model_used`: CharField - AI model used (optional)
|
||||
- `tokens_input`: IntegerField - Input tokens (optional)
|
||||
- `tokens_output`: IntegerField - Output tokens (optional)
|
||||
- `related_object_type`: CharField - Related object type (optional)
|
||||
- `related_object_id`: IntegerField - Related object ID (optional)
|
||||
- `metadata`: JSONField - Additional metadata
|
||||
- `created_at`: DateTimeField - Creation timestamp
|
||||
|
||||
### CreditService
|
||||
|
||||
**File**: `backend/igny8_core/modules/billing/services.py`
|
||||
|
||||
**Purpose**: Service for managing credits.
|
||||
|
||||
#### Methods
|
||||
|
||||
##### check_credits(account, required_credits)
|
||||
**Purpose**: Check if account has enough credits.
|
||||
|
||||
**Raises**: `InsufficientCreditsError` if account doesn't have enough credits
|
||||
|
||||
##### deduct_credits(account, amount, operation_type, description, ...)
|
||||
**Purpose**: Deduct credits and log transaction.
|
||||
|
||||
**Parameters**:
|
||||
- `account`: Account instance
|
||||
- `amount`: Number of credits to deduct
|
||||
- `operation_type`: Type of operation
|
||||
- `description`: Description of the transaction
|
||||
- `metadata`: Optional metadata dict
|
||||
- `cost_usd`: Optional cost in USD
|
||||
- `model_used`: Optional AI model used
|
||||
- `tokens_input`: Optional input tokens
|
||||
- `tokens_output`: Optional output tokens
|
||||
- `related_object_type`: Optional related object type
|
||||
- `related_object_id`: Optional related object ID
|
||||
|
||||
**Returns**: New credit balance
|
||||
|
||||
**Creates**:
|
||||
- `CreditTransaction` record
|
||||
- `CreditUsageLog` record
|
||||
|
||||
##### add_credits(account, amount, transaction_type, description, ...)
|
||||
**Purpose**: Add credits (purchase, subscription, etc.).
|
||||
|
||||
**Parameters**:
|
||||
- `account`: Account instance
|
||||
- `amount`: Number of credits to add
|
||||
- `transaction_type`: Type of transaction
|
||||
- `description`: Description of the transaction
|
||||
- `metadata`: Optional metadata dict
|
||||
|
||||
**Returns**: New credit balance
|
||||
|
||||
**Creates**: `CreditTransaction` record
|
||||
|
||||
##### calculate_credits_for_operation(operation_type, **kwargs)
|
||||
**Purpose**: Calculate credits needed for an operation.
|
||||
|
||||
**Parameters**:
|
||||
- `operation_type`: Type of operation
|
||||
- `**kwargs`: Operation-specific parameters
|
||||
|
||||
**Returns**: Number of credits required
|
||||
|
||||
**Raises**: `CreditCalculationError` if calculation fails
|
||||
|
||||
**Credit Costs**:
|
||||
- `clustering`: 1 credit per 30 keywords
|
||||
- `ideas`: 1 credit per idea
|
||||
- `content`: 3 credits per content piece
|
||||
- `images`: 1 credit per image
|
||||
- `reparse`: 1 credit per reparse
|
||||
|
||||
### Credit Integration
|
||||
|
||||
**Current Status**: Credits system is implemented but **not currently enforced** in AI functions.
|
||||
|
||||
**Future Implementation**: Credits should be checked and deducted before allowing AI operations.
|
||||
|
||||
---
|
||||
|
||||
## Plan Limits
|
||||
|
||||
### Planner Limits
|
||||
|
||||
- `max_keywords`: Total keywords allowed (not enforced)
|
||||
- `max_clusters`: Total clusters allowed (not enforced)
|
||||
- `max_content_ideas`: Total content ideas allowed (not enforced)
|
||||
- `daily_cluster_limit`: Max clusters per day (not enforced)
|
||||
- `monthly_cluster_ai_credits`: AI credits for clustering (not enforced)
|
||||
|
||||
### Writer Limits
|
||||
|
||||
- `daily_content_tasks`: Max content tasks per day (not enforced)
|
||||
- `daily_ai_requests`: Total AI executions per day (not enforced)
|
||||
- `monthly_word_count_limit`: Monthly word limit (not enforced)
|
||||
- `monthly_content_ai_credits`: AI credits for content (not enforced)
|
||||
|
||||
### Image Limits
|
||||
|
||||
- `monthly_image_count`: Max images per month (not enforced)
|
||||
- `daily_image_generation_limit`: Max images per day (not enforced)
|
||||
- `monthly_image_ai_credits`: AI credits for images (not enforced)
|
||||
- `max_images_per_task`: Max images per task (not enforced)
|
||||
- `image_model_choices`: Allowed image models (not enforced)
|
||||
|
||||
### Feature Flags
|
||||
|
||||
- `features`: JSON array of enabled features (e.g., `['ai_writer', 'image_gen', 'auto_publish']`)
|
||||
- **Not enforced**: Feature flags are not checked before allowing AI operations
|
||||
|
||||
### Plan Limits Enforcement
|
||||
|
||||
**Current Status**: Plan limits are defined but **not currently enforced** in AI functions.
|
||||
|
||||
**Future Implementation**: Plan limits should be checked before allowing operations.
|
||||
|
||||
---
|
||||
|
||||
## Multi-Tenancy Architecture
|
||||
|
||||
### Account Isolation
|
||||
|
||||
**Principle**: All data is isolated by account.
|
||||
|
||||
**Implementation**:
|
||||
1. **Model Level**: All models inherit `AccountBaseModel` (has `account` ForeignKey)
|
||||
2. **ViewSet Level**: All ViewSets inherit `AccountModelViewSet` (filters by `request.account`)
|
||||
3. **Middleware Level**: `AccountContextMiddleware` sets `request.account` from JWT token
|
||||
|
||||
**Access Control**:
|
||||
- Admin/Developer users: Bypass account filtering (see all accounts)
|
||||
- System account users: Bypass account filtering (see all accounts)
|
||||
- Regular users: Only see data from their account
|
||||
|
||||
### Site/Sector Hierarchy
|
||||
|
||||
**Structure**:
|
||||
```
|
||||
Account (1) ──< (N) Site
|
||||
Site (1) ──< (1-5) Sector
|
||||
Sector (1) ──< (N) Keywords, Clusters, ContentIdeas, Tasks
|
||||
```
|
||||
|
||||
**Implementation**:
|
||||
- Models inherit `SiteSectorBaseModel` (has `site` and `sector` ForeignKeys)
|
||||
- ViewSets inherit `SiteSectorModelViewSet` (filters by accessible sites)
|
||||
- User access control via `User.get_accessible_sites()`
|
||||
|
||||
**Site Access Control**:
|
||||
- System account users: All active sites
|
||||
- Developers: All active sites
|
||||
- Owners/Admins: All sites in their account
|
||||
- Editors/Viewers: Only sites granted via `SiteUserAccess`
|
||||
|
||||
### Data Isolation Flow
|
||||
|
||||
1. **Request Arrives**: JWT token contains account ID
|
||||
2. **Middleware**: `AccountContextMiddleware` extracts account ID and sets `request.account`
|
||||
3. **ViewSet**: `AccountModelViewSet.get_queryset()` filters by `request.account`
|
||||
4. **Database**: All queries automatically filtered by account
|
||||
|
||||
**Override**: Admin/Developer users bypass account filtering
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
The IGNY8 multi-tenancy architecture provides:
|
||||
|
||||
1. **Complete Account Isolation**: All data isolated by account with automatic filtering
|
||||
2. **Site/Sector Hierarchy**: Hierarchical organization with access control
|
||||
3. **Role-Based Access Control**: Granular permissions based on user roles
|
||||
4. **Plan-Based Limits**: Comprehensive plan limits (defined but not enforced)
|
||||
5. **Credits System**: Credit tracking and deduction system (implemented but not enforced)
|
||||
6. **Flexible Site Access**: Multiple sites can be active simultaneously
|
||||
7. **Industry Templates**: Global industry and sector templates
|
||||
|
||||
This architecture ensures data security, scalability, and flexibility while providing a solid foundation for subscription management and billing.
|
||||
|
||||
1420
docs/06-INFRASTRUCTURE-DEPLOYMENT.md
Normal file
1420
docs/06-INFRASTRUCTURE-DEPLOYMENT.md
Normal file
File diff suppressed because it is too large
Load Diff
274
docs/07-AI-FRAMEWORK.md
Normal file
274
docs/07-AI-FRAMEWORK.md
Normal file
@@ -0,0 +1,274 @@
|
||||
# IGNY8 AI Framework Documentation
|
||||
|
||||
**Version:** 1.0
|
||||
**Last Updated:** 2025-01-XX
|
||||
**Purpose:** Complete documentation of the unified AI framework architecture.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The IGNY8 AI Framework provides a unified, consistent architecture for all AI functions. It eliminates code duplication, standardizes progress tracking, and provides a single interface for all AI operations.
|
||||
|
||||
### Key Benefits
|
||||
|
||||
- **90% Code Reduction**: Functions are now ~100 lines instead of ~600
|
||||
- **Consistent UX**: All functions use the same progress modal and tracking
|
||||
- **Unified Logging**: Single `AITaskLog` table for all AI operations
|
||||
- **Easy Extension**: Add new functions by creating one class
|
||||
- **Better Debugging**: Detailed step-by-step tracking for all operations
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Directory Structure
|
||||
|
||||
```
|
||||
igny8_core/ai/
|
||||
├── __init__.py # Auto-registers all functions
|
||||
├── apps.py # Django app configuration
|
||||
├── admin.py # Admin interface for AITaskLog
|
||||
├── base.py # BaseAIFunction abstract class
|
||||
├── engine.py # AIEngine orchestrator
|
||||
├── processor.py # AIProcessor wrapper
|
||||
├── registry.py # Function registry
|
||||
├── tracker.py # StepTracker, ProgressTracker, CostTracker
|
||||
├── tasks.py # Unified Celery task entrypoint
|
||||
├── types.py # Shared dataclasses
|
||||
├── models.py # AITaskLog model
|
||||
└── functions/ # Function implementations
|
||||
├── __init__.py
|
||||
└── auto_cluster.py # Auto cluster function
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Core Components
|
||||
|
||||
### 1. BaseAIFunction
|
||||
|
||||
Abstract base class that all AI functions inherit from.
|
||||
|
||||
**Methods to implement:**
|
||||
- `get_name()`: Return function name
|
||||
- `prepare()`: Load and prepare data
|
||||
- `build_prompt()`: Build AI prompt
|
||||
- `parse_response()`: Parse AI response
|
||||
- `save_output()`: Save results to database
|
||||
|
||||
**Optional overrides:**
|
||||
- `validate()`: Custom validation
|
||||
- `get_max_items()`: Set item limit
|
||||
- `get_model()`: Specify AI model
|
||||
- `get_metadata()`: Function metadata
|
||||
|
||||
### 2. AIEngine
|
||||
|
||||
Central orchestrator that manages the execution pipeline.
|
||||
|
||||
**Phases:**
|
||||
- INIT (0-10%): Validation & setup
|
||||
- PREP (10-25%): Data loading & prompt building
|
||||
- AI_CALL (25-60%): API call to provider
|
||||
- PARSE (60-80%): Response parsing
|
||||
- SAVE (80-95%): Database operations
|
||||
- DONE (95-100%): Finalization
|
||||
|
||||
### 3. Function Registry
|
||||
|
||||
Dynamic function discovery system.
|
||||
|
||||
**Usage:**
|
||||
```python
|
||||
from igny8_core.ai.registry import register_function, get_function
|
||||
|
||||
# Register function
|
||||
register_function('auto_cluster', AutoClusterFunction)
|
||||
|
||||
# Get function
|
||||
fn = get_function('auto_cluster')
|
||||
```
|
||||
|
||||
### 4. Unified Celery Task
|
||||
|
||||
Single entrypoint for all AI functions.
|
||||
|
||||
**Endpoint:** `run_ai_task(function_name, payload, account_id)`
|
||||
|
||||
**Example:**
|
||||
```python
|
||||
from igny8_core.ai.tasks import run_ai_task
|
||||
|
||||
task = run_ai_task.delay(
|
||||
function_name='auto_cluster',
|
||||
payload={'ids': [1, 2, 3], 'sector_id': 1},
|
||||
account_id=1
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Function Implementation Example
|
||||
|
||||
### Auto Cluster Function
|
||||
|
||||
```python
|
||||
from igny8_core.ai.base import BaseAIFunction
|
||||
|
||||
class AutoClusterFunction(BaseAIFunction):
|
||||
def get_name(self) -> str:
|
||||
return 'auto_cluster'
|
||||
|
||||
def get_max_items(self) -> int:
|
||||
return 20
|
||||
|
||||
def prepare(self, payload: dict, account=None) -> Dict:
|
||||
# Load keywords
|
||||
ids = payload.get('ids', [])
|
||||
keywords = Keywords.objects.filter(id__in=ids)
|
||||
return {'keywords': keywords, ...}
|
||||
|
||||
def build_prompt(self, data: Dict, account=None) -> str:
|
||||
# Build clustering prompt
|
||||
return prompt_template.replace('[IGNY8_KEYWORDS]', keywords_text)
|
||||
|
||||
def parse_response(self, response: str, step_tracker=None) -> List[Dict]:
|
||||
# Parse AI response
|
||||
return clusters
|
||||
|
||||
def save_output(self, parsed, original_data, account, progress_tracker) -> Dict:
|
||||
# Save clusters to database
|
||||
return {'clusters_created': 5, 'keywords_updated': 20}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Endpoint Example
|
||||
|
||||
### Before (Old): ~300 lines
|
||||
|
||||
### After (New): ~50 lines
|
||||
|
||||
```python
|
||||
@action(detail=False, methods=['post'], url_path='auto_cluster')
|
||||
def auto_cluster(self, request):
|
||||
from igny8_core.ai.tasks import run_ai_task
|
||||
|
||||
account = getattr(request, 'account', None)
|
||||
account_id = account.id if account else None
|
||||
|
||||
payload = {
|
||||
'ids': request.data.get('ids', []),
|
||||
'sector_id': request.data.get('sector_id')
|
||||
}
|
||||
|
||||
task = run_ai_task.delay(
|
||||
function_name='auto_cluster',
|
||||
payload=payload,
|
||||
account_id=account_id
|
||||
)
|
||||
|
||||
return Response({
|
||||
'success': True,
|
||||
'task_id': str(task.id),
|
||||
'message': 'Clustering started'
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Progress Tracking
|
||||
|
||||
### Unified Progress Endpoint
|
||||
|
||||
**URL:** `/api/v1/system/settings/task_progress/<task_id>/`
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"state": "PROGRESS",
|
||||
"meta": {
|
||||
"phase": "AI_CALL",
|
||||
"percentage": 45,
|
||||
"message": "Analyzing keyword relationships...",
|
||||
"request_steps": [...],
|
||||
"response_steps": [...],
|
||||
"cost": 0.000123,
|
||||
"tokens": 1500
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Frontend Integration
|
||||
|
||||
All AI functions use the same progress modal:
|
||||
- Single `useProgressModal` hook
|
||||
- Unified progress endpoint
|
||||
- Consistent phase labels
|
||||
- Step-by-step logs
|
||||
|
||||
---
|
||||
|
||||
## Database Logging
|
||||
|
||||
### AITaskLog Model
|
||||
|
||||
Unified logging table for all AI operations.
|
||||
|
||||
**Fields:**
|
||||
- `task_id`: Celery task ID
|
||||
- `function_name`: Function name
|
||||
- `account`: Account (required)
|
||||
- `phase`: Current phase
|
||||
- `status`: success/error/pending
|
||||
- `cost`: API cost
|
||||
- `tokens`: Token usage
|
||||
- `request_steps`: Request step logs
|
||||
- `response_steps`: Response step logs
|
||||
- `error`: Error message (if any)
|
||||
|
||||
---
|
||||
|
||||
## Migration Guide
|
||||
|
||||
### Migrating Existing Functions
|
||||
|
||||
1. Create function class inheriting `BaseAIFunction`
|
||||
2. Implement required methods
|
||||
3. Register function in `ai/__init__.py`
|
||||
4. Update API endpoint to use `run_ai_task`
|
||||
5. Test and remove old code
|
||||
|
||||
### Example Migration
|
||||
|
||||
**Old code:**
|
||||
```python
|
||||
@action(...)
|
||||
def auto_cluster(self, request):
|
||||
# 300 lines of code
|
||||
```
|
||||
|
||||
**New code:**
|
||||
```python
|
||||
@action(...)
|
||||
def auto_cluster(self, request):
|
||||
# 20 lines using framework
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
The AI Framework provides:
|
||||
|
||||
1. **Unified Architecture**: Single framework for all AI functions
|
||||
2. **Code Reduction**: 90% less code per function
|
||||
3. **Consistent UX**: Same progress modal for all functions
|
||||
4. **Better Debugging**: Detailed step tracking
|
||||
5. **Easy Extension**: Add functions quickly
|
||||
6. **Unified Logging**: Single log table
|
||||
7. **Cost Tracking**: Automatic cost calculation
|
||||
|
||||
This architecture ensures maintainability, consistency, and extensibility while dramatically reducing code duplication.
|
||||
|
||||
128
docs/AI-ELEMENTS-TABLE.md
Normal file
128
docs/AI-ELEMENTS-TABLE.md
Normal file
@@ -0,0 +1,128 @@
|
||||
# AI-Related Elements Across Codebase
|
||||
|
||||
Complete table of all AI-related logic, configs, prompts, validations, retries, logging, response parsing, etc.
|
||||
|
||||
| Type | Name / Key | File | Function / Class | Line No | Notes |
|
||||
|------|------------|------|------------------|---------|-------|
|
||||
| **🧠 AI Core Functions** |
|
||||
| AI Core | `_auto_cluster_keywords_core` | `backend/igny8_core/modules/planner/tasks.py` | `_auto_cluster_keywords_core()` | 26 | Core clustering logic (legacy, used by old tasks) |
|
||||
| AI Core | `_generate_single_idea_core` | `backend/igny8_core/modules/planner/tasks.py` | `_generate_single_idea_core()` | 1047 | Core idea generation logic (legacy) |
|
||||
| AI Core | `auto_generate_content_task` | `backend/igny8_core/modules/writer/tasks.py` | `auto_generate_content_task()` | 27 | Celery task for content generation (legacy) |
|
||||
| AI Core | `AutoClusterFunction` | `backend/igny8_core/ai/functions/auto_cluster.py` | `AutoClusterFunction` class | 14 | New framework function for clustering |
|
||||
| AI Core | `cluster_keywords` | `backend/igny8_core/utils/ai_processor.py` | `AIProcessor.cluster_keywords()` | 1080 | Legacy clustering method |
|
||||
| AI Core | `generate_ideas` | `backend/igny8_core/utils/ai_processor.py` | `AIProcessor.generate_ideas()` | 1280 | Legacy idea generation method |
|
||||
| AI Core | `generate_content` | `backend/igny8_core/utils/ai_processor.py` | `AIProcessor.generate_content()` | 446 | Legacy content generation method |
|
||||
| AI Core | `generate_image` | `backend/igny8_core/utils/ai_processor.py` | `AIProcessor.generate_image()` | 656 | Image generation (OpenAI DALL-E / Runware) |
|
||||
| AI Core | `generate_image` | `backend/igny8_core/ai/processor.py` | `AIProcessor.generate_image()` | 61 | Framework wrapper for image generation |
|
||||
| AI Core | `run_ai_task` | `backend/igny8_core/ai/tasks.py` | `run_ai_task()` | 13 | Unified Celery entrypoint for all AI functions |
|
||||
| AI Core | `AIEngine.execute` | `backend/igny8_core/ai/engine.py` | `AIEngine.execute()` | 26 | Central orchestrator for all AI functions |
|
||||
| **🔁 Retry / Model Fallback Logic** |
|
||||
| Retry | `_call_openai` | `backend/igny8_core/utils/ai_processor.py` | `AIProcessor._call_openai()` | 125 | Main OpenAI API call method with error handling |
|
||||
| Retry | `_get_api_key` | `backend/igny8_core/utils/ai_processor.py` | `AIProcessor._get_api_key()` | 73 | Gets API key from IntegrationSettings or Django settings (fallback) |
|
||||
| Retry | `_get_model` | `backend/igny8_core/utils/ai_processor.py` | `AIProcessor._get_model()` | 98 | Gets model from IntegrationSettings or Django settings (fallback) |
|
||||
| Retry | `max_retries=3` | `backend/igny8_core/ai/tasks.py` | `@shared_task(bind=True, max_retries=3)` | 12 | Celery task retry configuration |
|
||||
| Retry | `max_retries=3` | `backend/igny8_core/modules/writer/tasks.py` | `@shared_task(bind=True, max_retries=3)` | 26 | Content generation task retry configuration |
|
||||
| Retry | Error handling | `backend/igny8_core/utils/ai_processor.py` | `_call_openai()` | 191-305 | HTTP error handling, JSON parsing errors, timeout handling |
|
||||
| Retry | Fallback to Django settings | `backend/igny8_core/utils/ai_processor.py` | `_get_api_key()` | 91-95 | Falls back to `OPENAI_API_KEY` or `RUNWARE_API_KEY` from settings |
|
||||
| Retry | Fallback to default model | `backend/igny8_core/utils/ai_processor.py` | `_get_model()` | 120-123 | Falls back to `DEFAULT_AI_MODEL` from settings (default: 'gpt-4.1') |
|
||||
| **🧱 Prompt Sources** |
|
||||
| Prompt | `clustering` | `backend/igny8_core/modules/system/utils.py` | `get_default_prompt('clustering')` | 10-30 | Hardcoded default clustering prompt template |
|
||||
| Prompt | `ideas` | `backend/igny8_core/modules/system/utils.py` | `get_default_prompt('ideas')` | 32-55 | Hardcoded default ideas generation prompt template |
|
||||
| Prompt | `content_generation` | `backend/igny8_core/modules/system/utils.py` | `get_default_prompt('content_generation')` | 57-75 | Hardcoded default content generation prompt template |
|
||||
| Prompt | `image_prompt_extraction` | `backend/igny8_core/modules/system/utils.py` | `get_default_prompt('image_prompt_extraction')` | 77-98 | Hardcoded default image prompt extraction template |
|
||||
| Prompt | `image_prompt_template` | `backend/igny8_core/modules/system/utils.py` | `get_default_prompt('image_prompt_template')` | 100 | Hardcoded default image prompt template |
|
||||
| Prompt | `negative_prompt` | `backend/igny8_core/modules/system/utils.py` | `get_default_prompt('negative_prompt')` | 102 | Hardcoded default negative prompt |
|
||||
| Prompt | `get_prompt_value` | `backend/igny8_core/modules/system/utils.py` | `get_prompt_value()` | 108 | Retrieves prompt from DB (AIPrompt model) or returns default |
|
||||
| Prompt | `AIPrompt` model | `backend/igny8_core/modules/system/models.py` | `AIPrompt` class | 13 | Database model storing account-specific prompts |
|
||||
| Prompt | `prompt_type` | `backend/igny8_core/modules/system/models.py` | `AIPrompt.prompt_type` | 25 | Choices: clustering, ideas, content_generation, image_prompt_extraction, image_prompt_template, negative_prompt |
|
||||
| Prompt | `build_prompt` | `backend/igny8_core/ai/functions/auto_cluster.py` | `AutoClusterFunction.build_prompt()` | 117 | Builds clustering prompt using `get_prompt_value()` |
|
||||
| Prompt | `[IGNY8_KEYWORDS]` | `backend/igny8_core/ai/functions/auto_cluster.py` | `build_prompt()` | 131 | Placeholder replaced with keyword list |
|
||||
| Prompt | `[IGNY8_CLUSTERS]` | `backend/igny8_core/modules/system/utils.py` | `get_default_prompt('ideas')` | 34 | Placeholder for ideas prompt |
|
||||
| Prompt | `[IGNY8_IDEA]` | `backend/igny8_core/modules/system/utils.py` | `get_default_prompt('content_generation')` | 60 | Placeholder for content generation prompt |
|
||||
| **⚠️ Validation / Limits** |
|
||||
| Validation | `validate` | `backend/igny8_core/ai/base.py` | `BaseAIFunction.validate()` | 34 | Base validation: checks for 'ids' array, max_items limit |
|
||||
| Validation | `validate` | `backend/igny8_core/ai/functions/auto_cluster.py` | `AutoClusterFunction.validate()` | 37 | Custom validation: checks keywords exist, plan limits |
|
||||
| Validation | `get_max_items` | `backend/igny8_core/ai/functions/auto_cluster.py` | `AutoClusterFunction.get_max_items()` | 34 | Returns 20 (max keywords per cluster operation) |
|
||||
| Validation | `check_credits` | `backend/igny8_core/modules/billing/services.py` | `CreditService.check_credits()` | 16 | Checks if account has enough credits |
|
||||
| Validation | Daily cluster limit | `backend/igny8_core/ai/functions/auto_cluster.py` | `validate()` | 59-71 | Checks `plan.daily_cluster_limit` against clusters created today |
|
||||
| Validation | Max clusters limit | `backend/igny8_core/ai/functions/auto_cluster.py` | `validate()` | 73-79 | Checks `plan.max_clusters` against total clusters |
|
||||
| Validation | `VALID_OPENAI_IMAGE_MODELS` | `backend/igny8_core/utils/ai_processor.py` | Constant | 34 | Set: {'dall-e-3', 'dall-e-2'} |
|
||||
| Validation | `VALID_SIZES_BY_MODEL` | `backend/igny8_core/utils/ai_processor.py` | Constant | 41 | Dict mapping models to valid sizes |
|
||||
| Validation | Model validation | `backend/igny8_core/utils/ai_processor.py` | `generate_image()` | 704-708 | Validates model is in VALID_OPENAI_IMAGE_MODELS |
|
||||
| Validation | Size validation | `backend/igny8_core/utils/ai_processor.py` | `generate_image()` | 719-724 | Validates size is valid for selected model |
|
||||
| Validation | Model rate validation | `backend/igny8_core/utils/ai_processor.py` | `_get_model()` | 112 | Validates model is in MODEL_RATES before using |
|
||||
| **🪵 AI Debug Steps** |
|
||||
| Debug Step | `addRequestStep` | `backend/igny8_core/ai/tracker.py` | `StepTracker.add_request_step()` | 21 | Adds request phase step (INIT, PREP, SAVE, DONE) |
|
||||
| Debug Step | `addResponseStep` | `backend/igny8_core/ai/tracker.py` | `StepTracker.add_response_step()` | 45 | Adds response phase step (AI_CALL, PARSE) |
|
||||
| Debug Step | `request_steps` | `backend/igny8_core/ai/tracker.py` | `StepTracker.request_steps` | 17 | List of request steps |
|
||||
| Debug Step | `response_steps` | `backend/igny8_core/ai/tracker.py` | `StepTracker.response_steps` | 18 | List of response steps |
|
||||
| Debug Step | `request_steps` | `backend/igny8_core/modules/planner/tasks.py` | `_auto_cluster_keywords_core()` | 37 | Legacy request steps tracking |
|
||||
| Debug Step | `response_steps` | `backend/igny8_core/modules/planner/tasks.py` | `_auto_cluster_keywords_core()` | 38 | Legacy response steps tracking |
|
||||
| Debug Step | `meta['request_steps']` | `backend/igny8_core/modules/system/integration_views.py` | `task_progress()` | 936-937 | Extracts request_steps from Celery task meta |
|
||||
| Debug Step | `meta['response_steps']` | `backend/igny8_core/modules/system/integration_views.py` | `task_progress()` | 938-939 | Extracts response_steps from Celery task meta |
|
||||
| Debug Step | `request_steps` | `backend/igny8_core/ai/models.py` | `AITaskLog.request_steps` | 31 | JSONField storing request steps in database |
|
||||
| Debug Step | `response_steps` | `backend/igny8_core/ai/models.py` | `AITaskLog.response_steps` | 32 | JSONField storing response steps in database |
|
||||
| Debug Step | `get_meta()` | `backend/igny8_core/ai/tracker.py` | `StepTracker.get_meta()` | 69 | Returns dict with request_steps and response_steps |
|
||||
| Debug Step | Step injection | `backend/igny8_core/ai/engine.py` | `execute()` | 47, 60, 68, 125, 138, 174 | Adds steps at each phase (INIT, PREP, AI_CALL, PARSE, SAVE, DONE) |
|
||||
| Debug Step | Step counter | `backend/igny8_core/ai/tracker.py` | `StepTracker.step_counter` | 19 | Auto-increments for sequential step numbers |
|
||||
| **🧾 Model Config / Selection** |
|
||||
| Model Config | `default_model` | `backend/igny8_core/utils/ai_processor.py` | `AIProcessor.__init__()` | 67 | Set from `_get_model()` or defaults to 'gpt-4.1' |
|
||||
| Model Config | `_get_model` | `backend/igny8_core/utils/ai_processor.py` | `AIProcessor._get_model()` | 98 | Gets model from IntegrationSettings.config['model'] or Django settings |
|
||||
| Model Config | `MODEL_RATES` | `backend/igny8_core/utils/ai_processor.py` | Constant | 19 | Dict: {'gpt-4.1': {...}, 'gpt-4o-mini': {...}, 'gpt-4o': {...}} |
|
||||
| Model Config | `IMAGE_MODEL_RATES` | `backend/igny8_core/utils/ai_processor.py` | Constant | 26 | Dict: {'dall-e-3': 0.040, 'dall-e-2': 0.020, ...} |
|
||||
| Model Config | `DEFAULT_AI_MODEL` | `backend/igny8_core/utils/ai_processor.py` | Django setting | 121 | Default: 'gpt-4.1' if not set |
|
||||
| Model Config | `get_model` | `backend/igny8_core/ai/base.py` | `BaseAIFunction.get_model()` | 70 | Override to specify model (defaults to account's default) |
|
||||
| Model Config | `model` parameter | `backend/igny8_core/ai/processor.py` | `call()` | 22 | Model can be passed to AIProcessor.call() |
|
||||
| Model Config | `json_models` | `backend/igny8_core/ai/processor.py` | `call()` | 40 | List: ['gpt-4o', 'gpt-4o-mini', 'gpt-4-turbo-preview'] (supports JSON mode) |
|
||||
| Model Config | `IntegrationSettings.config['model']` | `backend/igny8_core/modules/system/models.py` | `IntegrationSettings.config` | 56 | JSONField storing model name per account |
|
||||
| Model Config | Model selection | `backend/igny8_core/ai/engine.py` | `execute()` | 65 | Gets model via `fn.get_model(self.account)` |
|
||||
| **📦 Request/Response Structuring** |
|
||||
| Request/Response | `response_format` | `backend/igny8_core/utils/ai_processor.py` | `_call_openai()` | 131 | Optional dict for JSON mode: {'type': 'json_object'} |
|
||||
| Request/Response | `response_format` | `backend/igny8_core/ai/processor.py` | `call()` | 25 | Optional dict for JSON mode |
|
||||
| Request/Response | JSON mode auto-enable | `backend/igny8_core/ai/processor.py` | `call()` | 41-42 | Auto-enables JSON mode for json_models if not specified |
|
||||
| Request/Response | `_extract_json_from_response` | `backend/igny8_core/utils/ai_processor.py` | `AIProcessor._extract_json_from_response()` | 334 | Extracts JSON from response (handles markdown code blocks, multiline) |
|
||||
| Request/Response | `extract_json` | `backend/igny8_core/ai/processor.py` | `AIProcessor.extract_json()` | 57 | Wrapper for `_extract_json_from_response()` |
|
||||
| Request/Response | `parse_response` | `backend/igny8_core/ai/functions/auto_cluster.py` | `AutoClusterFunction.parse_response()` | 158 | Parses AI response into cluster data structure |
|
||||
| Request/Response | `parse_response` | `backend/igny8_core/ai/base.py` | `BaseAIFunction.parse_response()` | 75 | Abstract method for parsing AI response |
|
||||
| Request/Response | Request body format | `backend/igny8_core/utils/ai_processor.py` | `_call_openai()` | 171-183 | Format: {'model': str, 'messages': [...], 'temperature': float, 'max_tokens': int, 'response_format': dict} |
|
||||
| Request/Response | Response format | `backend/igny8_core/utils/ai_processor.py` | `_call_openai()` | 215-329 | Returns: {'content': str, 'input_tokens': int, 'output_tokens': int, 'total_tokens': int, 'model': str, 'cost': float, 'error': str, 'api_id': str} |
|
||||
| Request/Response | JSON parsing fallback | `backend/igny8_core/ai/functions/auto_cluster.py` | `parse_response()` | 173-176 | Falls back to `extract_json()` if direct JSON parse fails |
|
||||
| **📍 Paths / Constants** |
|
||||
| Path/Constant | `OPENAI_API_KEY` | `backend/igny8_core/utils/ai_processor.py` | Django setting | 93 | Fallback API key from Django settings |
|
||||
| Path/Constant | `RUNWARE_API_KEY` | `backend/igny8_core/utils/ai_processor.py` | Django setting | 95 | Fallback API key from Django settings |
|
||||
| Path/Constant | OpenAI API URL | `backend/igny8_core/utils/ai_processor.py` | `_call_openai()` | 163 | Hardcoded: 'https://api.openai.com/v1/chat/completions' |
|
||||
| Path/Constant | OpenAI Images URL | `backend/igny8_core/utils/ai_processor.py` | `generate_image()` | 735 | Hardcoded: 'https://api.openai.com/v1/images/generations' |
|
||||
| Path/Constant | Runware API URL | `backend/igny8_core/utils/ai_processor.py` | `generate_image()` | 844 | Hardcoded: 'https://api.runware.ai/v1' |
|
||||
| Path/Constant | `IntegrationSettings.config['apiKey']` | `backend/igny8_core/modules/system/models.py` | `IntegrationSettings.config` | 56 | JSONField storing API key per account |
|
||||
| Path/Constant | Prompt type choices | `backend/igny8_core/modules/system/models.py` | `AIPrompt.PROMPT_TYPE_CHOICES` | 16 | ['clustering', 'ideas', 'content_generation', 'image_prompt_extraction', 'image_prompt_template', 'negative_prompt'] |
|
||||
| Path/Constant | Integration type choices | `backend/igny8_core/modules/system/models.py` | `IntegrationSettings.INTEGRATION_TYPE_CHOICES` | 48 | ['openai', 'runware', 'gsc', 'image_generation'] |
|
||||
| Path/Constant | `get_function_instance` | `backend/igny8_core/ai/registry.py` | `get_function_instance()` | - | Gets AI function instance from registry |
|
||||
| Path/Constant | `auto_cluster` | `backend/igny8_core/ai/registry.py` | `register_lazy_function()` | 69 | Registered function name for clustering |
|
||||
| **💰 Cost Tracking** |
|
||||
| Cost | `CostTracker` | `backend/igny8_core/ai/tracker.py` | `CostTracker` class | 193 | Tracks API costs and token usage |
|
||||
| Cost | `record` | `backend/igny8_core/ai/tracker.py` | `CostTracker.record()` | 201 | Records cost, tokens, model for an operation |
|
||||
| Cost | Cost calculation | `backend/igny8_core/utils/ai_processor.py` | `_call_openai()` | 277-295 | Calculates cost from MODEL_RATES based on input/output tokens |
|
||||
| Cost | Image cost calculation | `backend/igny8_core/utils/ai_processor.py` | `generate_image()` | 790 | Calculates cost from IMAGE_MODEL_RATES * n images |
|
||||
| Cost | Credit logging | `backend/igny8_core/ai/engine.py` | `execute()` | 142-171 | Logs credit usage to CreditUsageLog after successful save |
|
||||
| Cost | `_calculate_credits_for_clustering` | `backend/igny8_core/ai/engine.py` | `_calculate_credits_for_clustering()` | 257 | Calculates credits used (from plan config or fallback formula) |
|
||||
| **📊 Progress Tracking** |
|
||||
| Progress | `ProgressTracker` | `backend/igny8_core/ai/tracker.py` | `ProgressTracker` class | 77 | Tracks progress updates for AI tasks |
|
||||
| Progress | `update` | `backend/igny8_core/ai/tracker.py` | `ProgressTracker.update()` | 89 | Updates Celery task state with progress |
|
||||
| Progress | Phase percentages | `backend/igny8_core/ai/engine.py` | `execute()` | 30-36 | INIT (0-10%), PREP (10-25%), AI_CALL (25-70%), PARSE (70-85%), SAVE (85-98%), DONE (98-100%) |
|
||||
| Progress | `update_ai_progress` | `backend/igny8_core/ai/tracker.py` | `ProgressTracker.update_ai_progress()` | 184 | Callback for AI processor progress updates |
|
||||
| **🗄️ Database Logging** |
|
||||
| Database | `AITaskLog` | `backend/igny8_core/ai/models.py` | `AITaskLog` model | 8 | Unified logging table for all AI tasks |
|
||||
| Database | `_log_to_database` | `backend/igny8_core/ai/engine.py` | `AIEngine._log_to_database()` | 220 | Logs task execution to AITaskLog table |
|
||||
| Database | Task logging | `backend/igny8_core/ai/engine.py` | `execute()` | 178 | Called after successful execution to log to database |
|
||||
| **🔄 Celery Integration** |
|
||||
| Celery | `run_ai_task` | `backend/igny8_core/ai/tasks.py` | `run_ai_task()` | 13 | Unified Celery task entrypoint |
|
||||
| Celery | Task state updates | `backend/igny8_core/ai/tracker.py` | `ProgressTracker.update()` | 124-131 | Updates Celery task state via `task.update_state()` |
|
||||
| Celery | Meta injection | `backend/igny8_core/modules/system/integration_views.py` | `task_progress()` | 936-991 | Extracts request_steps/response_steps from task meta and returns to frontend |
|
||||
| Celery | Error state | `backend/igny8_core/ai/tasks.py` | `run_ai_task()` | 68-84 | Updates task state to FAILURE on error |
|
||||
| **🔧 Utility Functions** |
|
||||
| Utility | `get_default_prompt` | `backend/igny8_core/modules/system/utils.py` | `get_default_prompt()` | 7 | Returns hardcoded default prompt by type |
|
||||
| Utility | `get_prompt_value` | `backend/igny8_core/modules/system/utils.py` | `get_prompt_value()` | 108 | Gets prompt from DB or returns default |
|
||||
| Utility | `check_moderation` | `backend/igny8_core/utils/ai_processor.py` | `AIProcessor.check_moderation()` | 594 | Checks content against OpenAI moderation API |
|
||||
| Utility | `prepare` | `backend/igny8_core/ai/functions/auto_cluster.py` | `AutoClusterFunction.prepare()` | 85 | Loads keywords with relationships |
|
||||
| Utility | `save_output` | `backend/igny8_core/ai/functions/auto_cluster.py` | `AutoClusterFunction.save_output()` | 205 | Saves clusters to database with transaction |
|
||||
|
||||
300
docs/ActiveDocs/AI-ELEMENTS-EXTRACTED.md
Normal file
300
docs/ActiveDocs/AI-ELEMENTS-EXTRACTED.md
Normal file
@@ -0,0 +1,300 @@
|
||||
# IGNY8 AI Elements Reference Table
|
||||
|
||||
Generated by extract_ai_elements.py analysis
|
||||
|
||||
---
|
||||
|
||||
## 🧠 AI Core Functions
|
||||
|
||||
| Function Name | Category | Type | File | Line | Uses AIProcessor | Celery | Progress | Steps | Prompt Source | Model Source |
|
||||
|---------------|----------|------|------|------|------------------|--------|----------|-------|---------------|--------------|
|
||||
| `_auto_cluster_keywords_core` | cluster | core_function | `backend/igny8_core/modules/planner/tasks.py` | 26 | ✅ | ❌ | ✅ | ✅ | Database (get_prompt_value) | AIProcessor.default_model |
|
||||
| `_generate_single_idea_core` | ideas | core_function | `backend/igny8_core/modules/planner/tasks.py` | 1047 | ✅ | ❌ | ✅ | ✅ | Database (get_prompt_value) | AIProcessor.default_model |
|
||||
| `auto_generate_content_task` | content | celery_task | `backend/igny8_core/modules/writer/tasks.py` | 27 | ✅ | ✅ | ✅ | ❌ | Database (get_prompt_value) | AIProcessor.default_model |
|
||||
| `auto_generate_images_task` | image | celery_task | `backend/igny8_core/modules/writer/tasks.py` | 741 | ✅ | ✅ | ✅ | ❌ | Database (get_prompt_value) | AIProcessor.default_model |
|
||||
| `AutoClusterFunction` | cluster | class | `backend/igny8_core/ai/functions/auto_cluster.py` | 14 | ✅ | ❌ | ✅ | ✅ | Database (get_prompt_value) | Function.get_model() |
|
||||
| `cluster_keywords` | cluster | method | `backend/igny8_core/utils/ai_processor.py` | 1080 | ✅ | ❌ | ✅ | ✅ | Inline/Hardcoded | AIProcessor.default_model |
|
||||
| `generate_ideas` | ideas | method | `backend/igny8_core/utils/ai_processor.py` | 1280 | ✅ | ❌ | ✅ | ✅ | Inline/Hardcoded | AIProcessor.default_model |
|
||||
| `generate_content` | content | method | `backend/igny8_core/utils/ai_processor.py` | 446 | ✅ | ❌ | ❌ | ❌ | Inline/Hardcoded | AIProcessor.default_model |
|
||||
| `generate_image` | image | method | `backend/igny8_core/utils/ai_processor.py` | 656 | ✅ | ❌ | ❌ | ❌ | Inline/Hardcoded | Parameter or default |
|
||||
| `run_ai_task` | unified | celery_task | `backend/igny8_core/ai/tasks.py` | 13 | ❌ | ✅ | ✅ | ✅ | Via function | Via function |
|
||||
| `execute` | unified | method | `backend/igny8_core/ai/engine.py` | 26 | ✅ | ❌ | ✅ | ✅ | Via function | Via function |
|
||||
|
||||
---
|
||||
|
||||
## 🧱 Prompt Sources
|
||||
|
||||
| Prompt Type | Source | File | Retrieval Method |
|
||||
|-------------|--------|------|------------------|
|
||||
| `clustering` | Hardcoded in get_default_prompt() | `backend/igny8_core/modules/system/utils.py` | `get_prompt_value()` -> AIPrompt model or default |
|
||||
| `ideas` | Hardcoded in get_default_prompt() | `backend/igny8_core/modules/system/utils.py` | `get_prompt_value()` -> AIPrompt model or default |
|
||||
| `content_generation` | Hardcoded in get_default_prompt() | `backend/igny8_core/modules/system/utils.py` | `get_prompt_value()` -> AIPrompt model or default |
|
||||
| `image_prompt_extraction` | Hardcoded in get_default_prompt() | `backend/igny8_core/modules/system/utils.py` | `get_prompt_value()` -> AIPrompt model or default |
|
||||
| `image_prompt_template` | Hardcoded in get_default_prompt() | `backend/igny8_core/modules/system/utils.py` | `get_prompt_value()` -> AIPrompt model or default |
|
||||
| `negative_prompt` | Hardcoded in get_default_prompt() | `backend/igny8_core/modules/system/utils.py` | `get_prompt_value()` -> AIPrompt model or default |
|
||||
|
||||
**Prompt Storage:**
|
||||
- **Database Model**: `AIPrompt` in `backend/igny8_core/modules/system/models.py`
|
||||
- **Table**: `igny8_ai_prompts`
|
||||
- **Fields**: `prompt_type`, `prompt_value`, `default_prompt`, `account` (FK)
|
||||
- **Unique Constraint**: `(account, prompt_type)`
|
||||
|
||||
**Prompt Retrieval Flow:**
|
||||
1. `get_prompt_value(account, prompt_type)` in `modules/system/utils.py:108`
|
||||
2. Tries: `AIPrompt.objects.get(account=account, prompt_type=prompt_type, is_active=True)`
|
||||
3. Falls back to: `get_default_prompt(prompt_type)` if not found
|
||||
|
||||
---
|
||||
|
||||
## 🧾 Model Configuration
|
||||
|
||||
| Model Name | Source | File | Selection Method |
|
||||
|------------|--------|------|------------------|
|
||||
| `gpt-4.1` | MODEL_RATES constant | `backend/igny8_core/utils/ai_processor.py` | `AIProcessor._get_model()` -> IntegrationSettings or Django settings |
|
||||
| `gpt-4o-mini` | MODEL_RATES constant | `backend/igny8_core/utils/ai_processor.py` | `AIProcessor._get_model()` -> IntegrationSettings or Django settings |
|
||||
| `gpt-4o` | MODEL_RATES constant | `backend/igny8_core/utils/ai_processor.py` | `AIProcessor._get_model()` -> IntegrationSettings or Django settings |
|
||||
| `dall-e-3` | IMAGE_MODEL_RATES constant | `backend/igny8_core/utils/ai_processor.py` | Parameter or default in `generate_image()` |
|
||||
| `dall-e-2` | IMAGE_MODEL_RATES constant | `backend/igny8_core/utils/ai_processor.py` | Parameter or default in `generate_image()` |
|
||||
|
||||
**Model Selection Flow:**
|
||||
1. `AIProcessor.__init__(account)` in `utils/ai_processor.py:54`
|
||||
2. Calls `_get_model('openai', account)` in `utils/ai_processor.py:98`
|
||||
3. Tries: `IntegrationSettings.objects.filter(integration_type='openai', account=account, is_active=True).first().config.get('model')`
|
||||
4. Validates model is in `MODEL_RATES` dict
|
||||
5. Falls back to: `settings.DEFAULT_AI_MODEL` (default: 'gpt-4.1')
|
||||
|
||||
**Model Storage:**
|
||||
- **Database Model**: `IntegrationSettings` in `backend/igny8_core/modules/system/models.py`
|
||||
- **Table**: `igny8_integration_settings`
|
||||
- **Fields**: `integration_type`, `config` (JSONField), `account` (FK)
|
||||
- **Config Structure**: `{"apiKey": "...", "model": "gpt-4.1", "enabled": true}`
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Validation & Limits
|
||||
|
||||
| Function | Validation Checks | Limit Checks |
|
||||
|----------|-------------------|--------------|
|
||||
| `_auto_cluster_keywords_core` | Has validate() call, Keywords exist check | Credit check, Plan limits (daily_cluster_limit, max_clusters) |
|
||||
| `AutoClusterFunction.validate()` | Base validation (ids array, max_items), Keywords exist | Plan limits (daily_cluster_limit, max_clusters) |
|
||||
| `auto_generate_content_task` | Task existence, Account validation | Credit check (via CreditService) |
|
||||
| `auto_generate_images_task` | Task existence, Account validation | Credit check (via CreditService) |
|
||||
| `generate_image` | Model validation (VALID_OPENAI_IMAGE_MODELS), Size validation (VALID_SIZES_BY_MODEL) | None |
|
||||
| `AIProcessor._get_model()` | Model in MODEL_RATES validation | None |
|
||||
|
||||
**Validation Details:**
|
||||
|
||||
1. **Plan Limits** (in `AutoClusterFunction.validate()`):
|
||||
- `plan.daily_cluster_limit` - Max clusters per day
|
||||
- `plan.max_clusters` - Total max clusters
|
||||
- Checked in `backend/igny8_core/ai/functions/auto_cluster.py:59-79`
|
||||
|
||||
2. **Credit Checks**:
|
||||
- `CreditService.check_credits(account, required_credits)` in `modules/billing/services.py:16`
|
||||
- Used before AI operations
|
||||
|
||||
3. **Model Validation**:
|
||||
- OpenAI images: Only `dall-e-3` and `dall-e-2` valid (line 704-708 in `ai_processor.py`)
|
||||
- Size validation per model (line 719-724 in `ai_processor.py`)
|
||||
|
||||
4. **Input Validation**:
|
||||
- Base validation in `BaseAIFunction.validate()` checks for 'ids' array and max_items limit
|
||||
- `AutoClusterFunction.get_max_items()` returns 20 (max keywords per cluster)
|
||||
|
||||
---
|
||||
|
||||
## 🔁 Retry & Error Handling
|
||||
|
||||
| Component | Retry Logic | Error Handling | Fallback |
|
||||
|-----------|-------------|----------------|----------|
|
||||
| `run_ai_task` | `max_retries=3` (Celery decorator) | Exception caught, task state updated to FAILURE | None |
|
||||
| `auto_generate_content_task` | `max_retries=3` (Celery decorator) | Try/except blocks, error logging | None |
|
||||
| `_call_openai` | None (single attempt) | HTTP error handling, JSON parse errors, timeout (60s) | Returns error dict |
|
||||
| `_get_api_key` | None | Exception caught, logs warning | Falls back to Django settings (`OPENAI_API_KEY`, `RUNWARE_API_KEY`) |
|
||||
| `_get_model` | None | Exception caught, logs warning | Falls back to Django settings (`DEFAULT_AI_MODEL`) |
|
||||
|
||||
---
|
||||
|
||||
## 🪵 AI Debug Steps
|
||||
|
||||
| Function | Request Steps | Response Steps | Step Tracking Method |
|
||||
|----------|---------------|----------------|---------------------|
|
||||
| `_auto_cluster_keywords_core` | ✅ (manual list) | ✅ (manual list) | Manual `request_steps.append()` and `response_steps.append()` |
|
||||
| `AutoClusterFunction` | ✅ (via StepTracker) | ✅ (via StepTracker) | `StepTracker.add_request_step()` and `add_response_step()` |
|
||||
| `run_ai_task` | ✅ (via engine) | ✅ (via engine) | Extracted from `engine.execute()` result |
|
||||
| `AIEngine.execute` | ✅ (via StepTracker) | ✅ (via StepTracker) | `StepTracker` instance tracks all steps |
|
||||
| `auto_generate_content_task` | ❌ | ❌ | No step tracking (legacy) |
|
||||
| `auto_generate_images_task` | ❌ | ❌ | No step tracking (legacy) |
|
||||
|
||||
**Step Tracking Implementation:**
|
||||
|
||||
1. **New Framework** (AIEngine):
|
||||
- Uses `StepTracker` class in `backend/igny8_core/ai/tracker.py`
|
||||
- Steps added at each phase: INIT, PREP, AI_CALL, PARSE, SAVE, DONE
|
||||
- Steps stored in `request_steps` and `response_steps` lists
|
||||
- Returned in result dict and logged to `AITaskLog` model
|
||||
|
||||
2. **Legacy Functions**:
|
||||
- Manual step tracking with lists
|
||||
- Steps added to `meta` dict for Celery task progress
|
||||
- Extracted in `integration_views.py:task_progress()` for frontend
|
||||
|
||||
3. **Step Structure**:
|
||||
```python
|
||||
{
|
||||
'stepNumber': int,
|
||||
'stepName': str, # INIT, PREP, AI_CALL, PARSE, SAVE, DONE
|
||||
'functionName': str,
|
||||
'status': 'success' | 'error' | 'pending',
|
||||
'message': str,
|
||||
'error': str (optional),
|
||||
'duration': int (milliseconds, optional)
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📦 Request/Response Structuring
|
||||
|
||||
| Function | Request Format | Response Format | JSON Mode | Parsing Method |
|
||||
|----------|----------------|-----------------|-----------|----------------|
|
||||
| `_call_openai` | OpenAI API format: `{'model': str, 'messages': [...], 'temperature': float, 'max_tokens': int, 'response_format': dict}` | `{'content': str, 'input_tokens': int, 'output_tokens': int, 'total_tokens': int, 'model': str, 'cost': float, 'error': str, 'api_id': str}` | ✅ (if `response_format={'type': 'json_object'}`) | `_extract_json_from_response()` |
|
||||
| `cluster_keywords` | Prompt string with keywords | JSON with 'clusters' array | ✅ (auto-enabled for json_models) | `_extract_json_from_response()` then extract 'clusters' |
|
||||
| `generate_ideas` | Prompt string with clusters | JSON with 'ideas' array | ✅ (auto-enabled for json_models) | `_extract_json_from_response()` then extract 'ideas' |
|
||||
| `generate_image` (OpenAI) | `{'prompt': str, 'model': str, 'n': int, 'size': str}` | `{'url': str, 'revised_prompt': str, 'cost': float}` | N/A | Direct JSON response |
|
||||
| `generate_image` (Runware) | Array format with `imageInference` tasks | `{'url': str, 'cost': float}` | N/A | Extract from nested response structure |
|
||||
|
||||
**JSON Mode Auto-Enable:**
|
||||
- Models: `['gpt-4o', 'gpt-4o-mini', 'gpt-4-turbo-preview']`
|
||||
- Auto-enabled in `AIProcessor.call()` if `response_format` not specified
|
||||
- Location: `backend/igny8_core/ai/processor.py:40-42`
|
||||
|
||||
**JSON Extraction:**
|
||||
- Primary: Direct `json.loads()` on response
|
||||
- Fallback: `_extract_json_from_response()` handles:
|
||||
- Markdown code blocks (```json ... ```)
|
||||
- Multiline JSON
|
||||
- Partial JSON extraction
|
||||
- Location: `backend/igny8_core/utils/ai_processor.py:334-440`
|
||||
|
||||
---
|
||||
|
||||
## 📍 Paths & Constants
|
||||
|
||||
| Constant | Value | File | Usage |
|
||||
|----------|-------|------|-------|
|
||||
| `OPENAI_API_KEY` | Django setting | `backend/igny8_core/utils/ai_processor.py:93` | Fallback API key |
|
||||
| `RUNWARE_API_KEY` | Django setting | `backend/igny8_core/utils/ai_processor.py:95` | Fallback API key |
|
||||
| `DEFAULT_AI_MODEL` | Django setting (default: 'gpt-4.1') | `backend/igny8_core/utils/ai_processor.py:121` | Fallback model |
|
||||
| OpenAI API URL | `'https://api.openai.com/v1/chat/completions'` | `backend/igny8_core/utils/ai_processor.py:163` | Text generation endpoint |
|
||||
| OpenAI Images URL | `'https://api.openai.com/v1/images/generations'` | `backend/igny8_core/utils/ai_processor.py:735` | Image generation endpoint |
|
||||
| Runware API URL | `'https://api.runware.ai/v1'` | `backend/igny8_core/utils/ai_processor.py:844` | Runware image generation |
|
||||
| `MODEL_RATES` | Dict with pricing per 1M tokens | `backend/igny8_core/utils/ai_processor.py:19` | Cost calculation |
|
||||
| `IMAGE_MODEL_RATES` | Dict with pricing per image | `backend/igny8_core/utils/ai_processor.py:26` | Image cost calculation |
|
||||
| `VALID_OPENAI_IMAGE_MODELS` | `{'dall-e-3', 'dall-e-2'}` | `backend/igny8_core/utils/ai_processor.py:34` | Model validation |
|
||||
| `VALID_SIZES_BY_MODEL` | Dict mapping models to valid sizes | `backend/igny8_core/utils/ai_processor.py:41` | Size validation |
|
||||
|
||||
---
|
||||
|
||||
## 💰 Cost Tracking
|
||||
|
||||
| Component | Cost Calculation | Token Tracking | Storage |
|
||||
|-----------|------------------|----------------|---------|
|
||||
| `_call_openai` | Calculated from `MODEL_RATES` based on input/output tokens | ✅ (input_tokens, output_tokens, total_tokens) | Returned in result dict |
|
||||
| `generate_image` (OpenAI) | `IMAGE_MODEL_RATES[model] * n` | N/A | Returned in result dict |
|
||||
| `generate_image` (Runware) | `0.036 * n` (hardcoded) | N/A | Returned in result dict |
|
||||
| `CostTracker` | Aggregates costs from multiple operations | ✅ (total_tokens) | In-memory during execution |
|
||||
| `AITaskLog` | Stored in `cost` field (DecimalField) | ✅ (stored in `tokens` field) | Database table `igny8_ai_task_logs` |
|
||||
| `CreditUsageLog` | Stored in `cost_usd` field | ✅ (tokens_input, tokens_output) | Database table (billing module) |
|
||||
|
||||
**Cost Calculation Formula:**
|
||||
```python
|
||||
# Text generation
|
||||
input_cost = (input_tokens / 1_000_000) * MODEL_RATES[model]['input']
|
||||
output_cost = (output_tokens / 1_000_000) * MODEL_RATES[model]['output']
|
||||
total_cost = input_cost + output_cost
|
||||
|
||||
# Image generation
|
||||
cost = IMAGE_MODEL_RATES[model] * n_images
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Progress Tracking
|
||||
|
||||
| Function | Progress Method | Phase Tracking | Percentage Mapping |
|
||||
|----------|-----------------|----------------|-------------------|
|
||||
| `_auto_cluster_keywords_core` | `progress_callback()` function | Manual phase strings | Manual percentage |
|
||||
| `auto_generate_content_task` | `self.update_state()` (Celery) | Manual phase strings | Manual percentage |
|
||||
| `AIEngine.execute` | `ProgressTracker.update()` | Automatic (INIT, PREP, AI_CALL, PARSE, SAVE, DONE) | Automatic: INIT (0-10%), PREP (10-25%), AI_CALL (25-70%), PARSE (70-85%), SAVE (85-98%), DONE (98-100%) |
|
||||
| `run_ai_task` | Via `AIEngine` | Via `AIEngine` | Via `AIEngine` |
|
||||
|
||||
**Progress Tracker:**
|
||||
- Class: `ProgressTracker` in `backend/igny8_core/ai/tracker.py:77`
|
||||
- Updates Celery task state via `task.update_state()`
|
||||
- Tracks: phase, percentage, message, current, total, meta
|
||||
|
||||
---
|
||||
|
||||
## 🗄️ Database Logging
|
||||
|
||||
| Component | Log Table | Fields Logged | When Logged |
|
||||
|-----------|-----------|---------------|-------------|
|
||||
| `AIEngine.execute` | `AITaskLog` | task_id, function_name, phase, message, status, duration, cost, tokens, request_steps, response_steps, error, payload, result | After execution (success or error) |
|
||||
| Credit usage | `CreditUsageLog` | account, operation_type, credits_used, cost_usd, model_used, tokens_input, tokens_output | After successful save operation |
|
||||
|
||||
**AITaskLog Model:**
|
||||
- Table: `igny8_ai_task_logs`
|
||||
- Location: `backend/igny8_core/ai/models.py:8`
|
||||
- Fields: All execution details including steps, costs, tokens, errors
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Celery Integration
|
||||
|
||||
| Task | Entrypoint | Task ID | State Updates | Error Handling |
|
||||
|------|------------|---------|---------------|----------------|
|
||||
| `run_ai_task` | `backend/igny8_core/ai/tasks.py:13` | `self.request.id` | Via `ProgressTracker` | Updates state to FAILURE, raises exception |
|
||||
| `auto_generate_content_task` | `backend/igny8_core/modules/writer/tasks.py:27` | `self.request.id` | Manual `self.update_state()` | Try/except, logs error |
|
||||
| `auto_generate_images_task` | `backend/igny8_core/modules/writer/tasks.py:741` | `self.request.id` | Manual `self.update_state()` | Try/except, logs error |
|
||||
|
||||
**Task Progress Endpoint:**
|
||||
- Route: `/api/v1/system/settings/task_progress/{task_id}/`
|
||||
- Handler: `IntegrationSettingsViewSet.task_progress()` in `modules/system/integration_views.py:936`
|
||||
- Extracts: `request_steps` and `response_steps` from task meta
|
||||
- Returns: Progress data to frontend for debug panel
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
**Key Findings:**
|
||||
|
||||
1. **Two AI Systems Coexist:**
|
||||
- **Legacy**: Direct functions in `modules/planner/tasks.py` and `modules/writer/tasks.py`
|
||||
- **New Framework**: `AIEngine` + `BaseAIFunction` classes in `ai/` directory
|
||||
|
||||
2. **Unified Entrypoint:**
|
||||
- `run_ai_task()` in `ai/tasks.py` is the unified Celery entrypoint
|
||||
- Uses `AIEngine` to execute any registered AI function
|
||||
|
||||
3. **Prompt Management:**
|
||||
- All prompts stored in `AIPrompt` model (database)
|
||||
- Fallback to hardcoded defaults in `get_default_prompt()`
|
||||
- Retrieved via `get_prompt_value(account, prompt_type)`
|
||||
|
||||
4. **Model Selection:**
|
||||
- Per-account via `IntegrationSettings.config['model']`
|
||||
- Falls back to Django `DEFAULT_AI_MODEL` setting
|
||||
- Validated against `MODEL_RATES` dict
|
||||
|
||||
5. **Step Tracking:**
|
||||
- New framework uses `StepTracker` class
|
||||
- Legacy functions use manual lists
|
||||
- Both stored in Celery task meta and `AITaskLog` model
|
||||
|
||||
6. **Cost Tracking:**
|
||||
- Calculated from `MODEL_RATES` and `IMAGE_MODEL_RATES`
|
||||
- Logged to `AITaskLog` and `CreditUsageLog`
|
||||
- Tracked via `CostTracker` during execution
|
||||
|
||||
121
docs/ActiveDocs/COLOR-TOKENS-COMPARISON.md
Normal file
121
docs/ActiveDocs/COLOR-TOKENS-COMPARISON.md
Normal file
@@ -0,0 +1,121 @@
|
||||
# Color Tokens Comparison
|
||||
|
||||
Side-by-side comparison of provided tokens vs. global color palette
|
||||
|
||||
| Color Token | Provided Value | Our Global Palette | Match Status | Notes |
|
||||
|-------------|----------------|-------------------|--------------|-------|
|
||||
| **RED** |
|
||||
| `--color-red-50` | `oklch(.971 .013 17.38)` | `oklch(0.971 0.013 17.38)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:10` |
|
||||
| `--color-red-100` | `oklch(.936 .032 17.717)` | `oklch(0.936 0.032 17.717)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:11` |
|
||||
| `--color-red-500` | `oklch(.637 .237 25.331)` | `oklch(0.637 0.237 25.331)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:15` |
|
||||
| `--color-red-600` | `oklch(.577 .245 27.325)` | `oklch(0.577 0.245 27.325)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:16` |
|
||||
| `--color-red-700` | `oklch(.505 .213 27.518)` | `oklch(0.505 0.213 27.518)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:17` |
|
||||
| **ORANGE** |
|
||||
| `--color-orange-50` | `#fff6ed` | `oklch(0.98 0.016 73.684)` ❌ | **DIFFERENT** | Provided: hex, Ours: oklch (in `tailwindcss/theme.css:22`) |
|
||||
| `--color-orange-100` | `#ffead5` | `oklch(0.954 0.038 75.164)` ❌ | **DIFFERENT** | Provided: hex, Ours: oklch (in `tailwindcss/theme.css:23`) |
|
||||
| `--color-orange-400` | `#fd853a` | `oklch(0.75 0.183 55.934)` ❌ | **DIFFERENT** | Provided: hex, Ours: oklch (in `tailwindcss/theme.css:26`) |
|
||||
| `--color-orange-500` | `#fb6514` | `oklch(0.705 0.213 47.604)` ❌ | **DIFFERENT** | Provided: hex, Ours: oklch (in `tailwindcss/theme.css:27`) |
|
||||
| `--color-orange-600` | `#ec4a0a` | `oklch(0.646 0.222 41.116)` ❌ | **DIFFERENT** | Provided: hex, Ours: oklch (in `tailwindcss/theme.css:28`) |
|
||||
| `--color-orange-700` | `#c4320a` | `oklch(0.553 0.195 38.402)` ❌ | **DIFFERENT** | Provided: hex, Ours: oklch (in `tailwindcss/theme.css:29`) |
|
||||
| **YELLOW** |
|
||||
| `--color-yellow-100` | `oklch(.973 .071 103.193)` | `oklch(0.973 0.071 103.193)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:47` |
|
||||
| `--color-yellow-600` | `oklch(.681 .162 75.834)` | `oklch(0.681 0.162 75.834)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:52` |
|
||||
| **GREEN** |
|
||||
| `--color-green-50` | `oklch(.982 .018 155.826)` | `oklch(0.982 0.018 155.826)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:70` |
|
||||
| `--color-green-100` | `oklch(.962 .044 156.743)` | `oklch(0.962 0.044 156.743)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:71` |
|
||||
| `--color-green-500` | `oklch(.723 .219 149.579)` | `oklch(0.723 0.219 149.579)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:75` |
|
||||
| `--color-green-600` | `oklch(.627 .194 149.214)` | `oklch(0.627 0.194 149.214)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:76` |
|
||||
| `--color-green-700` | `oklch(.527 .154 150.069)` | `oklch(0.527 0.154 150.069)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:77` |
|
||||
| **CYAN** |
|
||||
| `--color-cyan-100` | `oklch(.956 .045 203.388)` | `oklch(0.956 0.045 203.388)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:107` |
|
||||
| `--color-cyan-600` | `oklch(.609 .126 221.723)` | `oklch(0.609 0.126 221.723)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:112` |
|
||||
| **BLUE** |
|
||||
| `--color-blue-50` | `oklch(.97 .014 254.604)` | `oklch(0.97 0.014 254.604)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:130` |
|
||||
| `--color-blue-100` | `oklch(.932 .032 255.585)` | `oklch(0.932 0.032 255.585)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:131` |
|
||||
| `--color-blue-500` | `oklch(.623 .214 259.815)` | `oklch(0.623 0.214 259.815)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:135` |
|
||||
| `--color-blue-700` | `oklch(.488 .243 264.376)` | `oklch(0.488 0.243 264.376)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:137` |
|
||||
| **PURPLE** |
|
||||
| `--color-purple-50` | `oklch(.977 .014 308.299)` | `oklch(0.977 0.014 308.299)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:166` |
|
||||
| `--color-purple-100` | `oklch(.946 .033 307.174)` | `oklch(0.946 0.033 307.174)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:167` |
|
||||
| `--color-purple-400` | `oklch(.714 .203 305.504)` | `oklch(0.714 0.203 305.504)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:170` |
|
||||
| `--color-purple-500` | `oklch(.627 .265 303.9)` | `oklch(0.627 0.265 303.9)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:171` |
|
||||
| `--color-purple-600` | `oklch(.558 .288 302.321)` | `oklch(0.558 0.288 302.321)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:172` |
|
||||
| `--color-purple-700` | `oklch(.496 .265 301.924)` | `oklch(0.496 0.265 301.924)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:173` |
|
||||
| **PINK** |
|
||||
| `--color-pink-100` | `oklch(.948 .028 342.258)` | `oklch(0.948 0.028 342.258)` ✅ | **EXACT MATCH** | In `tailwindcss/theme.css:191` |
|
||||
| `--color-pink-600` | `oklch(.592 .249 .584)` | `oklch(0.592 0.249 0.584)` ⚠️ | **FORMAT DIFFERENCE** | Provided: `.584` (missing hue), Ours: full oklch (in `tailwindcss/theme.css:196`) |
|
||||
| **GRAY** |
|
||||
| `--color-gray-50` | `#f9fafb` | `#f9fafb` ✅ | **EXACT MATCH** | In `index.css:87` |
|
||||
| `--color-gray-100` | `#f2f4f7` | `#f2f4f7` ✅ | **EXACT MATCH** | In `index.css:88` |
|
||||
| `--color-gray-200` | `#e4e7ec` | `#e4e7ec` ✅ | **EXACT MATCH** | In `index.css:89` |
|
||||
| `--color-gray-300` | `#d0d5dd` | `#d0d5dd` ✅ | **EXACT MATCH** | In `index.css:90` |
|
||||
| `--color-gray-400` | `#98a2b3` | `#98a2b3` ✅ | **EXACT MATCH** | In `index.css:91` |
|
||||
| `--color-gray-500` | `#667085` | `#667085` ✅ | **EXACT MATCH** | In `index.css:92` |
|
||||
| `--color-gray-600` | `#475467` | `#475467` ✅ | **EXACT MATCH** | In `index.css:93` |
|
||||
| `--color-gray-700` | `#344054` | `#344054` ✅ | **EXACT MATCH** | In `index.css:94` |
|
||||
| `--color-gray-800` | `#1d2939` | `#1d2939` ✅ | **EXACT MATCH** | In `index.css:95` |
|
||||
| `--color-gray-900` | `#101828` | `#101828` ✅ | **EXACT MATCH** | In `index.css:96` |
|
||||
| `--color-gray-950` | `#0c111d` | `#0c111d` ✅ | **EXACT MATCH** | In `index.css:97` |
|
||||
| **BLACK & WHITE** |
|
||||
| `--color-black` | `#101828` | `#101828` ✅ | **EXACT MATCH** | In `index.css:58` |
|
||||
| `--color-white` | `#fff` | `#ffffff` ✅ | **EQUIVALENT** | In `index.css:57` (same color, different format) |
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
### ✅ **Matches (28 tokens)**
|
||||
- **Red**: All 5 tokens match exactly (oklch format)
|
||||
- **Yellow**: Both tokens match exactly (oklch format)
|
||||
- **Green**: All 5 tokens match exactly (oklch format)
|
||||
- **Cyan**: Both tokens match exactly (oklch format)
|
||||
- **Blue**: All 4 tokens match exactly (oklch format)
|
||||
- **Purple**: All 6 tokens match exactly (oklch format)
|
||||
- **Gray**: All 11 tokens match exactly (hex format)
|
||||
- **Black/White**: Both match (hex format)
|
||||
|
||||
### ❌ **Differences (6 tokens)**
|
||||
- **Orange**: All 6 tokens differ
|
||||
- Provided: Hex format (`#fff6ed`, `#ffead5`, etc.)
|
||||
- Ours: oklch format in Tailwind theme
|
||||
- **Note**: These are likely the same colors, just different formats. The hex values you provided match our custom orange palette in `index.css:100-111` (which uses hex), but Tailwind's theme uses oklch.
|
||||
|
||||
### ⚠️ **Format Issue (1 token)**
|
||||
- **Pink-600**: `oklch(.592 .249 .584)` - Missing hue value (should be 3 values: lightness, chroma, hue)
|
||||
|
||||
---
|
||||
|
||||
## Location of Colors in Codebase
|
||||
|
||||
### Tailwind Default Colors (oklch format)
|
||||
- **File**: `frontend/node_modules/tailwindcss/theme.css`
|
||||
- Contains: red, orange, yellow, green, cyan, blue, purple, pink (all in oklch)
|
||||
|
||||
### Custom IGNY8 Colors (hex format)
|
||||
- **File**: `frontend/src/index.css` (lines 55-154)
|
||||
- Contains:
|
||||
- Gray scale (hex)
|
||||
- Orange scale (hex) - **Matches your provided orange values!**
|
||||
- Brand colors (hex)
|
||||
- Success/Error/Warning colors (hex)
|
||||
|
||||
### IGNY8 Brand Colors
|
||||
- **File**: `frontend/src/styles/igny8-colors.css`
|
||||
- Contains: Custom brand colors with `--igny8-` prefix
|
||||
|
||||
---
|
||||
|
||||
## Recommendation
|
||||
|
||||
**Your provided orange tokens match our custom orange palette in `index.css`!**
|
||||
|
||||
The orange colors you provided are already in our codebase:
|
||||
- `--color-orange-50: #fff6ed` ✅ (line 101)
|
||||
- `--color-orange-100: #ffead5` ✅ (line 102)
|
||||
- `--color-orange-400: #fd853a` ✅ (line 105)
|
||||
- `--color-orange-500: #fb6514` ✅ (line 106)
|
||||
- `--color-orange-600: #ec4a0a` ✅ (line 107)
|
||||
- `--color-orange-700: #c4320a` ✅ (line 108)
|
||||
|
||||
All other colors (red, yellow, green, cyan, blue, purple, gray, black, white) also match either in Tailwind's theme or our custom palette.
|
||||
|
||||
1153
docs/MASTER-ARCHITECTURE.md
Normal file
1153
docs/MASTER-ARCHITECTURE.md
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user