salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: igny8 project

Browse Source
This commit is contained in:
igny8
2025-11-09 10:27:02 +00:00
commit 60b8188111
27265 changed files with 4360521 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

749
docs/ActiveDocs/01-ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,749 @@
# IGNY8 System Architecture
**Version:** 1.0
**Last Updated:** 2025-01-XX
**Purpose:** Complete system architecture documentation covering design patterns, principles, tech stack, and structural organization.
---
## Table of Contents
1. [System Overview](#system-overview)
2. [Tech Stack](#tech-stack)
3. [Core Architecture Principles](#core-architecture-principles)
4. [Project Structure](#project-structure)
5. [Key Architectural Patterns](#key-architectural-patterns)
6. [Multi-Tenancy Architecture](#multi-tenancy-architecture)
7. [Module Organization](#module-organization)
8. [API Architecture](#api-architecture)
9. [Frontend Architecture](#frontend-architecture)
10. [Backend Architecture](#backend-architecture)
11. [Database Architecture](#database-architecture)
12. [Security Architecture](#security-architecture)
13. [Deployment Architecture](#deployment-architecture)
---
## System Overview
**IGNY8** is a full-stack SaaS platform for SEO keyword management and AI-driven content generation. The system migrated from a WordPress plugin architecture to a modern Django + React architecture, providing a scalable, multi-account platform for content planning and generation.
### Core Capabilities
- **Multi-Account SaaS Platform**: Complete account isolation with site/sector hierarchy
- **SEO Keyword Management**: Import, organize, and cluster keywords
- **AI-Powered Content Planning**: Automated keyword clustering and content idea generation
- **AI Content Generation**: Automated blog post and article generation
- **Image Generation**: AI-powered image generation for content
- **WordPress Integration**: Publish content directly to WordPress sites
- **Subscription Management**: Plan-based limits and billing integration
---
## Tech Stack
### Backend
- **Framework**: Django 5.2+ with Django REST Framework (DRF)
- **Database**: PostgreSQL
- **Task Queue**: Celery with Redis broker
- **Authentication**: JWT (JSON Web Tokens) + Session-based auth
- **API**: RESTful API with DRF ViewSets
- **Caching**: Redis
- **File Storage**: Local filesystem (configurable for S3)
### Frontend
- **Framework**: React 19 with TypeScript
- **Build Tool**: Vite
- **Styling**: Tailwind CSS
- **State Management**: Zustand
- **Routing**: React Router v6
- **HTTP Client**: Fetch API with custom wrapper
- **UI Components**: Custom component library
### Infrastructure
- **Containerization**: Docker & Docker Compose
- **Reverse Proxy**: Caddy (HTTPS on port 443)
- **Process Management**: Supervisor (for Celery workers)
- **Monitoring**: Portainer (optional)
### Development Tools
- **Backend**: Python 3.11+, pip, virtualenv
- **Frontend**: Node.js 18+, npm/yarn
- **Version Control**: Git
- **Code Quality**: ESLint, Prettier (frontend), Black, Flake8 (backend)
---
## Core Architecture Principles
### 1. Configuration-Driven Everything
**Principle**: Zero HTML/JSX duplication - All UI rendered from configuration.
- **Tables, filters, forms** all driven by config files
- **Single source of truth** - Change config, UI updates everywhere
- **Page-local config** for page-specific settings
- **Shared snippets** for reusable column/filter/action definitions
**Implementation**:
- Frontend: Config files in `/config/pages/` and `/config/snippets/`
- Backend: DRF serializers and ViewSet actions
### 2. Multi-Tenancy Foundation
**Principle**: Complete account isolation with automatic filtering.
- All models inherit `AccountBaseModel` with automatic account isolation
- All ViewSets inherit `AccountModelViewSet` with automatic account filtering
- Middleware injects account context from JWT on every request
- Site > Sector hierarchy for content organization
**Implementation**:
- Base models: `AccountBaseModel`, `SiteSectorBaseModel`
- Base ViewSets: `AccountModelViewSet`, `SiteSectorModelViewSet`
- Middleware: `AccountContextMiddleware` sets `request.account`
### 3. Template System (4 Universal Templates)
**Principle**: Reusable templates for all page types.
- **DashboardTemplate**: Module home pages (KPIs, workflow steps, charts)
- **TablePageTemplate**: CRUD table pages (Keywords, Clusters, Tasks, etc.)
- **FormPageTemplate**: Settings/form pages (Settings, Integration, etc.)
- **SystemPageTemplate**: System/admin pages (Logs, Status, Monitoring)
**Implementation**:
- Frontend: `/templates/` directory with 4 template components
- Config-driven: Templates accept config objects for customization
### 4. Unified AI Processor
**Principle**: Single interface for all AI operations.
- Single `AIProcessor` class handles all AI operations
- Manual and automated workflows use same functions
- Action-based routing: 'clustering', 'ideas', 'content_generation', 'image_generation'
- Account-specific API keys and model configuration
**Implementation**:
- Backend: `AIProcessor` class in `/utils/ai_processor.py`
- Integration: Loads API keys from `IntegrationSettings` model
### 5. Module-Based Organization
**Principle**: Clear module boundaries with shared utilities.
- Each module = Django app (`/modules/{module}/`)
- Clear module boundaries with shared utilities
- Module router pattern for subpage routing
- Consistent structure across modules
**Implementation**:
- Backend: `/modules/planner/`, `/modules/writer/`, `/modules/system/`, `/modules/billing/`
- Frontend: `/pages/Planner/`, `/pages/Writer/`, `/pages/Settings/`, etc.
---
## Project Structure
```
igny8/
├── backend/ # Django backend
│ └── igny8_core/ # Django project
│ ├── auth/ # Multi-tenancy, User, Account, Plan models
│ │ ├── models.py # Account, User, Plan, Site, Sector, Industry models
│ │ ├── views.py # Account, User, Site, Sector ViewSets
│ │ ├── serializers.py # Account, User, Plan serializers
│ │ └── urls.py # Auth module URLs
│ ├── modules/ # Feature modules
│ │ ├── planner/ # Keywords, Clusters, Ideas
│ │ │ ├── models.py # Keywords, Clusters, ContentIdeas models
│ │ │ ├── views.py # KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
│ │ │ ├── tasks.py # Celery tasks for AI operations
│ │ │ ├── serializers.py # Model serializers
│ │ │ └── urls.py # Planner module URLs
│ │ ├── writer/ # Tasks, Content, Images
│ │ │ ├── models.py # Tasks, Content, Images models
│ │ │ ├── views.py # TasksViewSet
│ │ │ ├── tasks.py # Celery tasks for content/image generation
│ │ │ └── urls.py # Writer module URLs
│ │ ├── system/ # Settings, Prompts, Integration
│ │ │ ├── models.py # AIPrompt, IntegrationSettings, AuthorProfile, Strategy
│ │ │ ├── views.py # AIPromptViewSet, AuthorProfileViewSet
│ │ │ ├── integration_views.py # IntegrationSettingsViewSet, task_progress
│ │ │ ├── utils.py # Default prompts, prompt loading
│ │ │ └── urls.py # System module URLs
│ │ └── billing/ # Credits, Transactions, Usage
│ │ ├── models.py # CreditTransaction, UsageLog models
│ │ ├── views.py # Billing ViewSets
│ │ └── services.py # CreditService
│ ├── api/ # API base classes
│ │ └── base.py # AccountModelViewSet, SiteSectorModelViewSet
│ ├── utils/ # Shared utilities
│ │ ├── ai_processor.py # Unified AI interface
│ │ └── content_normalizer.py # Content processing utilities
│ ├── middleware/ # Custom middleware
│ │ ├── account.py # AccountContextMiddleware (sets request.account)
│ │ └── resource_tracker.py # ResourceTrackerMiddleware (API metrics)
│ ├── settings.py # Django settings
│ ├── urls.py # Root URL configuration
│ └── celery.py # Celery configuration
│
├── frontend/ # React frontend
│ └── src/
│ ├── pages/ # Page components
│ │ ├── Planner/ # KeywordsPage, ClustersPage, IdeasPage, Dashboard
│ │ ├── Writer/ # TasksPage, DraftsPage, PublishedPage, Dashboard
│ │ ├── Settings/ # General, Integration, Status, ImportExport
│ │ ├── Billing/ # Credits, Transactions, Usage
│ │ └── AuthPages/ # SignIn, SignUp
│ ├── templates/ # 4 master templates
│ │ ├── DashboardTemplate.tsx
│ │ ├── TablePageTemplate.tsx
│ │ ├── FormPageTemplate.tsx
│ │ └── SystemPageTemplate.tsx
│ ├── components/ # UI components
│ │ ├── layout/ # AppLayout, Sidebar, Header, Breadcrumbs
│ │ ├── table/ # DataTable, Filters, Actions, Pagination
│ │ ├── ui/ # Button, Card, Modal, Toast, etc.
│ │ └── auth/ # ProtectedRoute, Auth components
│ ├── config/ # Configuration files
│ │ ├── pages/ # Page-specific configs
│ │ │ └── keywords.config.tsx
│ │ ├── snippets/ # Shared column/filter/action definitions
│ │ │ ├── columns.snippets.ts
│ │ │ ├── filters.snippets.ts
│ │ │ └── actions.snippets.ts
│ │ └── routes.config.ts # Route configuration
│ ├── store/ # Zustand stores
│ │ ├── authStore.ts # Authentication state
│ │ ├── plannerStore.ts # Planner module state
│ │ ├── siteStore.ts # Site selection state
│ │ └── aiRequestLogsStore.ts # AI request/response logs
│ ├── services/ # API clients
│ │ └── api.ts # fetchAPI, API functions
│ ├── hooks/ # Custom React hooks
│ │ ├── useProgressModal.ts # Progress modal for long-running tasks
│ │ └── useAuth.ts # Authentication hook
│ ├── layout/ # Layout components
│ │ └── AppLayout.tsx # Main app layout wrapper
│ ├── App.tsx # Root component with routing
│ └── main.tsx # Entry point
│
└── docs/ # Documentation
└── ActiveDocs/ # Active documentation
├── 01-ARCHITECTURE.md # This file
├── 02-FRONTEND.md # Frontend documentation
├── 03-BACKEND.md # Backend documentation
├── 04-AI-FUNCTIONS.md # AI functions documentation
└── 05-ACCOUNT-USER-PLAN.md # Account/User/Plan documentation
```
---
## Key Architectural Patterns
### Configuration System (Page-Local Config + Shared Snippets)
**Rule**: Config = Page-Local, Snippets = Shared
**Structure**:
```
/pages/Planner/KeywordsPage.tsx
├── Imports snippets from /config/snippets/
├── Defines page-local tableConfig, filterConfig, actionsConfig
└── Passes config to TablePageTemplate
/config/snippets/
├── columns.snippets.ts # statusColumn, titleColumn, etc.
├── filters.snippets.ts # statusFilter, dateRangeFilter, etc.
├── actions.snippets.ts # commonActions, bulkActions, etc.
```
**Benefits**:
- Reusable components across pages
- Page-specific customization
- Single source of truth for shared definitions
### Base ViewSet Pattern
**Backend Pattern**: All ViewSets inherit from base classes for consistent behavior.
**Base Classes**:
- `AccountModelViewSet`: Automatic account filtering
- `SiteSectorModelViewSet`: Account + site/sector filtering + access control
**Benefits**:
- Consistent access control
- Automatic account isolation
- Reduced code duplication
### Celery Task Pattern
**Pattern**: Long-running operations use Celery tasks with progress tracking.
**Structure**:
1. API endpoint queues Celery task
2. Task updates progress via `update_state()`
3. Frontend polls `task_progress` endpoint
4. Progress displayed in modal
**Benefits**:
- Non-blocking API responses
- Real-time progress updates
- Scalable background processing
### AI Function Pattern
**Pattern**: All AI functions follow consistent structure.
**Structure**:
1. API Endpoint: Validates input, queues Celery task
2. Celery Task: Wraps core function with progress tracking
3. Core Function: Business logic, calls AIProcessor
4. AIProcessor: Makes API calls, returns structured data
5. Core Function: Saves results to database
**Benefits**:
- Consistent error handling
- Progress tracking
- Reusable AI interface
---
## Multi-Tenancy Architecture
### Account Isolation
**Principle**: All data is isolated by account.
**Implementation**:
- All models inherit `AccountBaseModel` (has `account` ForeignKey)
- All ViewSets inherit `AccountModelViewSet` (filters by `request.account`)
- Middleware sets `request.account` from JWT token
**Access Control**:
- Admin/Developer users: Bypass account filtering (see all accounts)
- System account users: Bypass account filtering (see all accounts)
- Regular users: Only see data from their account
### Site/Sector Hierarchy
**Structure**:
```
Account (1) ──< (N) Site
Site (1) ──< (1-5) Sector
Sector (1) ──< (N) Keywords, Clusters, ContentIdeas, Tasks
```
**Implementation**:
- Models inherit `SiteSectorBaseModel` (has `site` and `sector` ForeignKeys)
- ViewSets inherit `SiteSectorModelViewSet` (filters by accessible sites)
- User access control via `User.get_accessible_sites()`
**Site Access Control**:
- System account users: All active sites
- Developers: All active sites
- Owners/Admins: All sites in their account
- Editors/Viewers: Only sites granted via `SiteUserAccess`
---
## Module Organization
### Planner Module
**Purpose**: Keyword management and content planning.
**Models**:
- `Keywords`: Individual keywords with volume, difficulty, intent
- `Clusters`: Keyword clusters (groups of related keywords)
- `ContentIdeas`: Content ideas generated from clusters
**ViewSets**:
- `KeywordViewSet`: CRUD + `auto_cluster` action
- `ClusterViewSet`: CRUD + `auto_generate_ideas` action
- `ContentIdeasViewSet`: CRUD operations
**Tasks**:
- `auto_cluster_keywords_task`: AI-powered keyword clustering
- `auto_generate_ideas_task`: AI-powered content idea generation
### Writer Module
**Purpose**: Content generation and management.
**Models**:
- `Tasks`: Content generation tasks
- `Content`: Generated content (HTML)
- `Images`: Generated images for tasks
**ViewSets**:
- `TasksViewSet`: CRUD + `auto_generate_content`, `auto_generate_images` actions
**Tasks**:
- `auto_generate_content_task`: AI-powered content generation
- `auto_generate_images_task`: AI-powered image generation
### System Module
**Purpose**: System settings, prompts, and integrations.
**Models**:
- `AIPrompt`: AI prompt templates (clustering, ideas, content, images)
- `IntegrationSettings`: API keys and configuration (OpenAI, Runware, etc.)
- `AuthorProfile`: Writing style profiles
- `Strategy`: Content strategies per sector
**ViewSets**:
- `AIPromptViewSet`: CRUD for prompts
- `IntegrationSettingsViewSet`: CRUD + `test_openai`, `test_runware`, `generate_image`, `task_progress` actions
- `AuthorProfileViewSet`: CRUD for author profiles
- `StrategyViewSet`: CRUD for strategies
### Billing Module
**Purpose**: Credits, transactions, and usage tracking.
**Models**:
- `CreditTransaction`: Credit purchase/usage transactions
- `UsageLog`: Daily/monthly usage tracking
**ViewSets**:
- `CreditTransactionViewSet`: CRUD for transactions
- `UsageLogViewSet`: Read-only usage logs
**Services**:
- `CreditService`: Credit calculation and deduction logic
---
## API Architecture
### RESTful API Design
**Base URL**: `/api/v1/`
**Endpoint Structure**:
- `/api/v1/planner/keywords/` - Keywords CRUD
- `/api/v1/planner/keywords/auto_cluster/` - Auto-cluster action
- `/api/v1/planner/clusters/` - Clusters CRUD
- `/api/v1/planner/clusters/auto_generate_ideas/` - Auto-generate ideas action
- `/api/v1/writer/tasks/` - Tasks CRUD
- `/api/v1/writer/tasks/auto_generate_content/` - Auto-generate content action
- `/api/v1/system/settings/task_progress/{task_id}/` - Task progress polling
### Authentication
**Methods**:
- JWT (JSON Web Tokens) - Primary method
- Session-based auth - Fallback for admin
**Flow**:
1. User signs in → Backend returns JWT token
2. Frontend stores token in localStorage
3. Frontend includes token in `Authorization: Bearer {token}` header
4. Backend middleware validates token and sets `request.user` and `request.account`
### Response Format
**Success Response**:
```json
{
"success": true,
"data": { ... },
"message": "Optional message"
}
```
**Error Response**:
```json
{
"success": false,
"message": "Error message",
"errors": { ... }
}
```
### Pagination
**Format**: Page-based pagination
**Response**:
```json
{
"count": 100,
"next": "http://api.example.com/api/v1/resource/?page=2",
"previous": null,
"results": [ ... ]
}
```
---
## Frontend Architecture
### Component Hierarchy
```
App
└── AppLayout
├── Sidebar (navigation)
├── Header (user menu, notifications)
└── Main Content
└── Page Component
└── Template (DashboardTemplate, TablePageTemplate, etc.)
└── Components (DataTable, Filters, etc.)
```
### State Management
**Zustand Stores**:
- `authStore`: Authentication state (user, token, account)
- `plannerStore`: Planner module state
- `siteStore`: Selected site/sector
- `aiRequestLogsStore`: AI request/response logs
- `pageSizeStore`: Table page size preference
**Local State**: React `useState` for component-specific state
### Routing
**Structure**: React Router v6 with nested routes
**Routes**:
- `/` - Home/Dashboard
- `/planner` - Planner Dashboard
- `/planner/keywords` - Keywords page
- `/planner/clusters` - Clusters page
- `/planner/ideas` - Ideas page
- `/writer` - Writer Dashboard
- `/writer/tasks` - Tasks page
- `/settings` - Settings pages
**Protected Routes**: All routes except `/signin` and `/signup` require authentication
---
## Backend Architecture
### Model Inheritance Hierarchy
```
models.Model
└── AccountBaseModel (adds account ForeignKey)
└── SiteSectorBaseModel (adds site, sector ForeignKeys)
└── Keywords, Clusters, ContentIdeas, Tasks, etc.
```
### ViewSet Inheritance Hierarchy
```
viewsets.ModelViewSet
└── AccountModelViewSet (adds account filtering)
└── SiteSectorModelViewSet (adds site/sector filtering)
└── KeywordViewSet, ClusterViewSet, TasksViewSet, etc.
```
### Middleware Stack
1. **SecurityMiddleware**: Django security middleware
2. **SessionMiddleware**: Session management
3. **AuthenticationMiddleware**: User authentication
4. **AccountContextMiddleware**: Sets `request.account` from JWT
5. **ResourceTrackerMiddleware**: Tracks API request metrics
### Celery Task Architecture
**Broker**: Redis
**Workers**: Separate Celery worker processes
**Task Structure**:
```python
@shared_task(bind=True)
def my_task(self, ...):
# Update progress
self.update_state(state='PROGRESS', meta={...})
# Do work
result = do_work()
# Return result
return result
```
**Progress Tracking**:
- Frontend polls `/api/v1/system/settings/task_progress/{task_id}/`
- Backend returns task state and meta information
- Progress displayed in modal
---
## Database Architecture
### Core Tables
- `igny8_accounts`: Account information
- `igny8_users`: User accounts
- `igny8_plans`: Subscription plans
- `igny8_subscriptions`: Active subscriptions
- `igny8_sites`: Sites within accounts
- `igny8_sectors`: Sectors within sites
- `igny8_industries`: Global industry templates
- `igny8_industry_sectors`: Industry sector templates
### Planner Tables
- `igny8_keywords`: Keywords
- `igny8_clusters`: Keyword clusters
- `igny8_content_ideas`: Content ideas
### Writer Tables
- `igny8_tasks`: Content generation tasks
- `igny8_content`: Generated content
- `igny8_images`: Generated images
### System Tables
- `igny8_ai_prompts`: AI prompt templates
- `igny8_integration_settings`: API keys and configuration
- `igny8_author_profiles`: Writing style profiles
- `igny8_strategies`: Content strategies
### Billing Tables
- `igny8_credit_transactions`: Credit transactions
- `igny8_usage_logs`: Usage tracking
### Indexes
**Account Isolation**: All tables have indexes on `account`
**Site/Sector Filtering**: Tables with site/sector have composite indexes on `(account, site, sector)`
**Performance**: Indexes on frequently queried fields (status, created_at, etc.)
---
## Security Architecture
### Authentication
**JWT Tokens**:
- Signed with secret key
- Contains user ID and account ID
- Expires after configured time
- Stored in localStorage (frontend)
**Session Auth**:
- Fallback for admin interface
- Django session framework
### Authorization
**Role-Based Access Control (RBAC)**:
- `developer`: Full system access
- `owner`: Full account access
- `admin`: Account admin access
- `editor`: Content editing access
- `viewer`: Read-only access
**Access Control**:
- Account-level: Automatic filtering by `request.account`
- Site-level: Filtering by `user.get_accessible_sites()`
- Action-level: Permission checks in ViewSet actions
### Data Isolation
**Account Isolation**:
- All queries filtered by account
- Admin/Developer override for system accounts
**Site Access Control**:
- Users can only access granted sites
- Admin/Developer override for all sites
### API Security
**CORS**: Configured for frontend domain
**CSRF**: Enabled for session-based auth
**Rate Limiting**: (Future implementation)
**Input Validation**: DRF serializers validate all input
---
## Deployment Architecture
### Docker Compose Setup
**Services**:
- `backend`: Django application (port 8010/8011)
- `frontend`: React application (port 5173/8021)
- `db`: PostgreSQL database
- `redis`: Redis for Celery broker and caching
- `caddy`: Reverse proxy (HTTPS on port 443)
- `celery-worker`: Celery worker process
- `celery-beat`: Celery beat scheduler (optional)
### Environment Configuration
**Backend**:
- `DJANGO_SETTINGS_MODULE`: Django settings module
- `DATABASE_URL`: PostgreSQL connection string
- `REDIS_URL`: Redis connection string
- `SECRET_KEY`: Django secret key
- `OPENAI_API_KEY`: OpenAI API key (fallback)
**Frontend**:
- `VITE_API_URL`: Backend API URL
- `VITE_APP_NAME`: Application name
### Scaling Considerations
**Horizontal Scaling**:
- Multiple Celery workers
- Multiple backend instances (load balanced)
- Multiple frontend instances (static files)
**Vertical Scaling**:
- Database connection pooling
- Redis connection pooling
- Celery worker concurrency
### Monitoring
**Application Monitoring**:
- ResourceTrackerMiddleware tracks API request metrics
- Celery task monitoring via Flower (optional)
**Infrastructure Monitoring**:
- Portainer for container monitoring
- Database monitoring via PostgreSQL logs
- Redis monitoring via Redis CLI
---
## Summary
The IGNY8 architecture is built on:
1. **Configuration-Driven Design**: Zero duplication, single source of truth
2. **Multi-Tenancy Foundation**: Complete account isolation with site/sector hierarchy
3. **Template System**: 4 universal templates for all page types
4. **Unified AI Interface**: Single AIProcessor for all AI operations
5. **Module-Based Organization**: Clear boundaries with shared utilities
6. **RESTful API**: Consistent API design with DRF
7. **Modern Frontend**: React + TypeScript with Zustand state management
8. **Scalable Backend**: Django + Celery for async processing
9. **Security First**: JWT auth, RBAC, data isolation
10. **Docker Deployment**: Containerized for easy deployment and scaling
This architecture ensures scalability, maintainability, and extensibility while maintaining a clean separation of concerns across modules.

776
docs/ActiveDocs/02-FRONTEND.md Normal file
View File

@@ -0,0 +1,776 @@
# IGNY8 Frontend Documentation
**Version:** 1.0
**Last Updated:** 2025-01-XX
**Purpose:** Complete frontend documentation covering pages, components, templates, routing, state management, and configuration system.
---
## Table of Contents
1. [Frontend Overview](#frontend-overview)
2. [Tech Stack](#tech-stack)
3. [Project Structure](#project-structure)
4. [Routing System](#routing-system)
5. [Template System](#template-system)
6. [Component Library](#component-library)
7. [State Management](#state-management)
8. [API Integration](#api-integration)
9. [Configuration System](#configuration-system)
10. [Pages](#pages)
11. [Hooks](#hooks)
12. [Utilities](#utilities)
---
## Frontend Overview
The IGNY8 frontend is a React 19 application built with TypeScript, using Vite as the build tool and Tailwind CSS for styling. The frontend follows a configuration-driven architecture where UI components are rendered from configuration objects, eliminating code duplication and ensuring consistency.
### Key Features
- **Configuration-Driven UI**: All tables, filters, and forms driven by config files
- **4 Universal Templates**: DashboardTemplate, TablePageTemplate, FormPageTemplate, SystemPageTemplate
- **TypeScript**: Full type safety across the application
- **Zustand State Management**: Lightweight, performant state management
- **React Router v6**: Modern routing with nested routes
- **Responsive Design**: Mobile-first approach with Tailwind CSS
---
## Tech Stack
### Core Technologies
- **React 19**: UI library
- **TypeScript**: Type safety
- **Vite**: Build tool and dev server
- **Tailwind CSS**: Utility-first CSS framework
- **React Router v6**: Client-side routing
### State Management
- **Zustand**: Lightweight state management library
- **localStorage**: Persistence for Zustand stores
### HTTP Client
- **Fetch API**: Native browser API with custom wrapper
- **Auto-retry**: Automatic retry on network failures
- **Token refresh**: Automatic JWT token refresh
### UI Components
- **Custom Component Library**: Built-in components (Button, Card, Modal, etc.)
- **Icons**: Custom icon library
- **Toast Notifications**: Toast notification system
---
## Project Structure
```
frontend/src/
├── pages/ # Page components
│ ├── Planner/ # Planner module pages
│ │ ├── Dashboard.tsx
│ │ ├── Keywords.tsx
│ │ ├── Clusters.tsx
│ │ └── Ideas.tsx
│ ├── Writer/ # Writer module pages
│ │ ├── Dashboard.tsx
│ │ ├── Tasks.tsx
│ │ ├── Content.tsx
│ │ ├── Drafts.tsx
│ │ ├── Images.tsx
│ │ └── Published.tsx
│ ├── Settings/ # Settings pages
│ │ ├── General.tsx
│ │ ├── Users.tsx
│ │ ├── Sites.tsx
│ │ ├── Integration.tsx
│ │ └── ...
│ ├── AuthPages/ # Authentication pages
│ │ ├── SignIn.tsx
│ │ └── SignUp.tsx
│ └── ...
├── templates/ # 4 universal templates
│ ├── DashboardTemplate.tsx
│ ├── TablePageTemplate.tsx
│ ├── FormPageTemplate.tsx
│ └── SystemPageTemplate.tsx
├── components/ # UI components
│ ├── layout/ # Layout components
│ │ ├── AppLayout.tsx
│ │ ├── Sidebar.tsx
│ │ └── Header.tsx
│ ├── table/ # Table components
│ │ ├── DataTable.tsx
│ │ ├── Filters.tsx
│ │ └── Pagination.tsx
│ ├── ui/ # UI primitives
│ │ ├── button/
│ │ ├── card/
│ │ ├── modal/
│ │ └── ...
│ └── auth/ # Auth components
│ └── ProtectedRoute.tsx
├── config/ # Configuration files
│ ├── pages/ # Page-specific configs
│ │ └── keywords.config.tsx
│ ├── snippets/ # Shared snippets
│ │ ├── columns.snippets.ts
│ │ ├── filters.snippets.ts
│ │ └── actions.snippets.ts
│ └── routes.config.ts # Route configuration
├── store/ # Zustand stores
│ ├── authStore.ts # Authentication state
│ ├── plannerStore.ts # Planner module state
│ ├── siteStore.ts # Site selection state
│ ├── sectorStore.ts # Sector selection state
│ ├── aiRequestLogsStore.ts # AI request/response logs
│ └── pageSizeStore.ts # Table page size preference
├── services/ # API clients
│ └── api.ts # fetchAPI, API functions
├── hooks/ # Custom React hooks
│ ├── useProgressModal.ts # Progress modal for long-running tasks
│ └── useAuth.ts # Authentication hook
├── layout/ # Layout components
│ └── AppLayout.tsx # Main app layout wrapper
├── utils/ # Utility functions
│ └── difficulty.ts # Difficulty utilities
├── App.tsx # Root component with routing
└── main.tsx # Entry point
```
---
## Routing System
### Route Structure
**File**: `App.tsx`
**Public Routes**:
- `/signin` - Sign in page
- `/signup` - Sign up page
**Protected Routes** (require authentication):
- `/` - Home/Dashboard
- `/planner` - Planner Dashboard
- `/planner/keywords` - Keywords page
- `/planner/clusters` - Clusters page
- `/planner/ideas` - Ideas page
- `/writer` - Writer Dashboard
- `/writer/tasks` - Tasks page
- `/writer/content` - Content page
- `/writer/drafts` - Drafts page
- `/writer/images` - Images page
- `/writer/published` - Published page
- `/thinker` - Thinker Dashboard
- `/thinker/prompts` - Prompts page
- `/thinker/author-profiles` - Author Profiles page
- `/thinker/strategies` - Strategies page
- `/thinker/image-testing` - Image Testing page
- `/billing/credits` - Credits page
- `/billing/transactions` - Transactions page
- `/billing/usage` - Usage page
- `/settings` - Settings pages
- `/analytics` - Analytics page
- `/schedules` - Schedules page
### Route Configuration
**File**: `config/routes.config.ts`
Defines route metadata including:
- Route paths
- Labels for navigation
- Icons
- Breadcrumb labels
- Nested route structure
### Protected Routes
**Component**: `components/auth/ProtectedRoute.tsx`
**Functionality**:
- Checks authentication status from `authStore`
- Redirects to `/signin` if not authenticated
- Wraps protected routes with `AppLayout`
---
## Template System
### 1. DashboardTemplate
**Purpose**: Module home pages with KPIs, workflow steps, and charts.
**Usage**:
```tsx
<DashboardTemplate
title="Planner Dashboard"
subtitle="Manage your SEO keywords and content planning"
metrics={metrics}
workflowSteps={steps}
charts={charts}
/>
```
**Features**:
- Header metrics (KPIs)
- Workflow steps
- Charts and visualizations
- Quick actions
### 2. TablePageTemplate
**Purpose**: CRUD table pages (Keywords, Clusters, Tasks, etc.).
**Usage**:
```tsx
<TablePageTemplate
title="Keywords"
subtitle="Manage and organize SEO keywords"
columns={columns}
data={data}
filters={filterConfig}
onSort={(field, direction) => ...}
onEdit={(row) => ...}
onDelete={(row) => ...}
bulkActions={bulkActions}
/>
```
**Features**:
- Data table with sorting
- Filters (text, select, date range, custom)
- Pagination
- Bulk actions
- Row actions (edit, delete)
- AI Request/Response Logs section
- Import/Export functionality
**Configuration**:
- `columns`: Column definitions (key, label, sortable, render, etc.)
- `filters`: Filter definitions (type, options, custom render)
- `bulkActions`: Bulk action definitions
- `actions`: Row action definitions
### 3. FormPageTemplate
**Purpose**: Settings/form pages (Settings, Integration, etc.).
**Usage**:
```tsx
<FormPageTemplate
title="Settings"
subtitle="Configure your account settings"
sections={sections}
onSubmit={(data) => ...}
/>
```
**Features**:
- Form sections
- Form validation
- Save/Cancel buttons
- Success/Error notifications
### 4. SystemPageTemplate
**Purpose**: System/admin pages (Logs, Status, Monitoring).
**Usage**:
```tsx
<SystemPageTemplate
title="System Status"
subtitle="Monitor system health and performance"
content={content}
/>
```
**Features**:
- System information display
- Logs viewer
- Status indicators
- Performance metrics
---
## Component Library
### Layout Components
#### AppLayout
**File**: `layout/AppLayout.tsx`
**Purpose**: Main app layout wrapper.
**Features**:
- Sidebar navigation
- Header with user menu
- Main content area
- Breadcrumbs
#### Sidebar
**File**: `components/layout/Sidebar.tsx`
**Purpose**: Navigation sidebar.
**Features**:
- Module navigation
- Active route highlighting
- Collapsible sections
#### Header
**File**: `components/layout/Header.tsx`
**Purpose**: Top header bar.
**Features**:
- User menu
- Notifications
- Site/Sector selector
### Table Components
#### DataTable
**File**: `components/table/DataTable.tsx`
**Purpose**: Data table component.
**Features**:
- Sortable columns
- Selectable rows
- Row actions
- Responsive design
#### Filters
**File**: `components/table/Filters.tsx`
**Purpose**: Filter component.
**Features**:
- Text filters
- Select filters
- Date range filters
- Custom filters
#### Pagination
**File**: `components/ui/pagination/CompactPagination.tsx`
**Purpose**: Pagination component.
**Features**:
- Page navigation
- Page size selector
- Total count display
### UI Components
#### Button
**File**: `components/ui/button/Button.tsx`
**Variants**: primary, secondary, danger, ghost, link
**Sizes**: sm, md, lg
#### Card
**File**: `components/ui/card/Card.tsx`
**Purpose**: Card container component.
#### Modal
**File**: `components/ui/modal/Modal.tsx`
**Purpose**: Modal dialog component.
**Variants**: FormModal, ProgressModal, AlertModal
#### Toast
**File**: `components/ui/toast/ToastContainer.tsx`
**Purpose**: Toast notification system.
**Types**: success, error, warning, info
#### Input
**File**: `components/form/input/InputField.tsx`
**Purpose**: Text input component.
#### Select
**File**: `components/form/SelectDropdown.tsx`
**Purpose**: Select dropdown component.
#### Checkbox
**File**: `components/form/input/Checkbox.tsx`
**Purpose**: Checkbox component.
### Auth Components
#### ProtectedRoute
**File**: `components/auth/ProtectedRoute.tsx`
**Purpose**: Route protection component.
**Functionality**:
- Checks authentication
- Redirects to signin if not authenticated
- Wraps children with AppLayout
---
## State Management
### Zustand Stores
#### authStore
**File**: `store/authStore.ts`
**State**:
- `user`: Current user object
- `token`: JWT access token
- `refreshToken`: JWT refresh token
- `isAuthenticated`: Authentication status
- `loading`: Loading state
**Actions**:
- `login(email, password)`: Sign in user
- `logout()`: Sign out user
- `register(data)`: Register new user
- `setUser(user)`: Set user object
- `setToken(token)`: Set access token
- `refreshToken()`: Refresh access token
**Persistence**: localStorage (persisted)
#### plannerStore
**File**: `store/plannerStore.ts`
**State**: Planner module-specific state
**Actions**: Planner module actions
#### siteStore
**File**: `store/siteStore.ts`
**State**:
- `activeSite`: Currently selected site
- `sites`: List of accessible sites
**Actions**:
- `setActiveSite(site)`: Set active site
- `loadSites()`: Load accessible sites
**Persistence**: localStorage (persisted)
#### sectorStore
**File**: `store/sectorStore.ts`
**State**:
- `activeSector`: Currently selected sector
- `sectors`: List of sectors for active site
**Actions**:
- `setActiveSector(sector)`: Set active sector
- `loadSectorsForSite(siteId)`: Load sectors for site
**Persistence**: localStorage (persisted)
#### aiRequestLogsStore
**File**: `store/aiRequestLogsStore.ts`
**State**:
- `logs`: Array of AI request/response logs
**Actions**:
- `addLog(log)`: Add new log entry
- `addRequestStep(logId, step)`: Add request step to log
- `addResponseStep(logId, step)`: Add response step to log
- `updateLog(logId, data)`: Update log entry
- `clearLogs()`: Clear all logs
**Purpose**: Tracks AI function execution with step-by-step logs
#### pageSizeStore
**File**: `store/pageSizeStore.ts`
**State**:
- `pageSize`: Table page size preference
**Actions**:
- `setPageSize(size)`: Set page size
**Persistence**: localStorage (persisted)
---
## API Integration
### API Service
**File**: `services/api.ts`
**Functions**:
- `fetchAPI(url, options)`: Generic API fetch wrapper
- `fetchKeywords(filters)`: Fetch keywords
- `createKeyword(data)`: Create keyword
- `updateKeyword(id, data)`: Update keyword
- `deleteKeyword(id)`: Delete keyword
- `bulkDeleteKeywords(ids)`: Bulk delete keywords
- `autoClusterKeywords(ids)`: Auto-cluster keywords
- `fetchClusters(filters)`: Fetch clusters
- `autoGenerateIdeas(clusterIds)`: Auto-generate ideas
- `fetchTasks(filters)`: Fetch tasks
- `autoGenerateContent(taskIds)`: Auto-generate content
- `autoGenerateImages(taskIds)`: Auto-generate images
- And more...
**Features**:
- Automatic JWT token inclusion
- Automatic token refresh on 401
- Auto-retry on network failures
- Error handling
- Request/response logging
### API Base URL
**Auto-detection**:
- Checks environment variables (`VITE_BACKEND_URL`, `VITE_API_URL`)
- Falls back to auto-detection based on current origin
- Supports localhost, IP addresses, and production subdomain
**Default**: `https://api.igny8.com/api`
---
## Configuration System
### Page-Local Config
**Location**: `config/pages/`
**Example**: `keywords.config.tsx`
**Structure**:
```tsx
export const createKeywordsPageConfig = () => ({
columns: [...],
filters: [...],
bulkActions: [...],
actions: [...],
});
```
**Usage**:
```tsx
import { createKeywordsPageConfig } from '../../config/pages/keywords.config';
const config = createKeywordsPageConfig();
```
### Shared Snippets
**Location**: `config/snippets/`
#### columns.snippets.ts
**Purpose**: Reusable column definitions.
**Examples**:
- `statusColumn`: Status column with badge
- `titleColumn`: Title column with link
- `dateColumn`: Date column with formatting
#### filters.snippets.ts
**Purpose**: Reusable filter definitions.
**Examples**:
- `statusFilter`: Status dropdown filter
- `dateRangeFilter`: Date range filter
- `searchFilter`: Text search filter
#### actions.snippets.ts
**Purpose**: Reusable action definitions.
**Examples**:
- `commonActions`: Edit, Delete actions
- `bulkActions`: Bulk delete, bulk update actions
### Route Configuration
**File**: `config/routes.config.ts`
**Structure**:
```tsx
export const routes: RouteConfig[] = [
{
path: '/planner',
label: 'Planner',
icon: 'Planner',
children: [
{ path: '/planner/keywords', label: 'Keywords' },
...
],
},
...
];
```
**Functions**:
- `getBreadcrumbs(pathname)`: Get breadcrumbs for current route
---
## Pages
### Planner Module
#### Keywords Page
**File**: `pages/Planner/Keywords.tsx`
**Features**:
- Keyword CRUD operations
- Auto-cluster functionality
- Import/Export (CSV)
- Filters (status, cluster, intent, difficulty, volume)
- Bulk actions (delete, status update)
- AI Request/Response Logs
**Configuration**: Uses `keywords.config.tsx`
#### Clusters Page
**File**: `pages/Planner/Clusters.tsx`
**Features**:
- Cluster CRUD operations
- Auto-generate ideas functionality
- Filters (status, sector)
- Bulk actions
#### Ideas Page
**File**: `pages/Planner/Ideas.tsx`
**Features**:
- Content ideas CRUD operations
- Filters (status, cluster, content type)
- Bulk actions
### Writer Module
#### Tasks Page
**File**: `pages/Writer/Tasks.tsx`
**Features**:
- Task CRUD operations
- Auto-generate content functionality
- Auto-generate images functionality
- Filters (status, cluster, content type)
- Bulk actions
### Settings Pages
#### Sites Page
**File**: `pages/Settings/Sites.tsx`
**Features**:
- Site CRUD operations
- Site activation/deactivation
- Multiple sites can be active simultaneously
#### Integration Page
**File**: `pages/Settings/Integration.tsx`
**Features**:
- Integration settings (OpenAI, Runware)
- API key configuration
- Test connections
- Image generation testing
---
## Hooks
### useProgressModal
**File**: `hooks/useProgressModal.ts`
**Purpose**: Progress modal for long-running Celery tasks.
**Usage**:
```tsx
const progressModal = useProgressModal();
// Start task
progressModal.start(taskId, 'Task started...');
// Task progress is automatically polled
// Modal displays progress updates
```
**Features**:
- Automatic polling of task progress
- Progress percentage display
- Step-by-step logs (request/response steps)
- Success/Error handling
- Auto-close on completion
### useAuth
**File**: `hooks/useAuth.ts`
**Purpose**: Authentication hook.
**Usage**:
```tsx
const { user, isAuthenticated, login, logout } = useAuth();
```
**Features**:
- Access to auth state
- Login/logout functions
- Authentication status
---
## Utilities
### Difficulty Utilities
**File**: `utils/difficulty.ts`
**Functions**:
- `getDifficultyLabelFromNumber(number)`: Get difficulty label (Easy, Medium, Hard)
- `getDifficultyRange()`: Get difficulty range options
### API Utilities
**File**: `services/api.ts`
**Helper Functions**:
- `getActiveSiteId()`: Get active site ID from store
- `getActiveSectorId()`: Get active sector ID from store
- `getAuthToken()`: Get JWT token from store
- `getRefreshToken()`: Get refresh token from store
---
## Summary
The IGNY8 frontend is built on:
1. **Configuration-Driven Architecture**: Zero duplication, single source of truth
2. **4 Universal Templates**: Reusable templates for all page types
3. **TypeScript**: Full type safety
4. **Zustand State Management**: Lightweight, performant state management
5. **React Router v6**: Modern routing with nested routes
6. **Component Library**: Reusable UI components
7. **API Integration**: Automatic token handling and error management
8. **Responsive Design**: Mobile-first with Tailwind CSS
This architecture ensures consistency, maintainability, and extensibility while providing a great user experience.

860
docs/ActiveDocs/03-BACKEND.md Normal file
View File

@@ -0,0 +1,860 @@
# IGNY8 Backend Documentation
**Version:** 1.0
**Last Updated:** 2025-01-XX
**Purpose:** Complete backend documentation covering models, views, APIs, modules, serializers, tasks, and structure.
---
## Table of Contents
1. [Backend Overview](#backend-overview)
2. [Tech Stack](#tech-stack)
3. [Project Structure](#project-structure)
4. [Models](#models)
5. [ViewSets](#viewsets)
6. [Serializers](#serializers)
7. [Celery Tasks](#celery-tasks)
8. [API Endpoints](#api-endpoints)
9. [Base Classes](#base-classes)
10. [Middleware](#middleware)
11. [Utilities](#utilities)
12. [Modules](#modules)
---
## Backend Overview
The IGNY8 backend is a Django 5.2+ application using Django REST Framework (DRF) for API endpoints. The backend follows a modular architecture with clear separation of concerns, automatic account isolation, and support for asynchronous task processing via Celery.
### Key Features
- **Multi-Tenancy**: Complete account isolation with automatic filtering
- **RESTful API**: DRF ViewSets with consistent response format
- **Celery Integration**: Asynchronous task processing for long-running operations
- **Account/Site/Sector Hierarchy**: Hierarchical data organization
- **AI Integration**: Unified AIProcessor for all AI operations
- **Progress Tracking**: Real-time progress updates for Celery tasks
---
## Tech Stack
### Core Technologies
- **Django 5.2+**: Web framework
- **Django REST Framework**: API framework
- **PostgreSQL**: Database
- **Celery**: Asynchronous task queue
- **Redis**: Celery broker and caching
### Key Libraries
- **django-filter**: Advanced filtering
- **djangorestframework-simplejwt**: JWT authentication
- **requests**: HTTP client for external APIs
- **python-dotenv**: Environment variable management
---
## Project Structure
```
backend/igny8_core/
├── auth/ # Multi-tenancy and authentication
│ ├── models.py # Account, User, Plan, Site, Sector, Industry models
│ ├── views.py # Account, User, Site, Sector ViewSets
│ ├── serializers.py # Account, User, Plan serializers
│ └── urls.py # Auth module URLs
├── modules/ # Feature modules
│ ├── planner/ # Keywords, Clusters, Ideas
│ │ ├── models.py # Keywords, Clusters, ContentIdeas models
│ │ ├── views.py # KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
│ │ ├── tasks.py # Celery tasks for AI operations
│ │ ├── serializers.py # Model serializers
│ │ └── urls.py # Planner module URLs
│ ├── writer/ # Tasks, Content, Images
│ │ ├── models.py # Tasks, Content, Images models
│ │ ├── views.py # TasksViewSet
│ │ ├── tasks.py # Celery tasks for content/image generation
│ │ └── urls.py # Writer module URLs
│ ├── system/ # Settings, Prompts, Integration
│ │ ├── models.py # AIPrompt, IntegrationSettings, AuthorProfile, Strategy
│ │ ├── views.py # AIPromptViewSet, AuthorProfileViewSet
│ │ ├── integration_views.py # IntegrationSettingsViewSet, task_progress
│ │ ├── utils.py # Default prompts, prompt loading
│ │ └── urls.py # System module URLs
│ └── billing/ # Credits, Transactions, Usage
│ ├── models.py # CreditTransaction, UsageLog models
│ ├── views.py # Billing ViewSets
│ └── services.py # CreditService
├── api/ # API base classes
│ ├── base.py # AccountModelViewSet, SiteSectorModelViewSet
│ └── pagination.py # CustomPageNumberPagination
├── utils/ # Shared utilities
│ ├── ai_processor.py # Unified AI interface
│ └── content_normalizer.py # Content processing utilities
├── middleware/ # Custom middleware
│ ├── account.py # AccountContextMiddleware (sets request.account)
│ └── resource_tracker.py # ResourceTrackerMiddleware (API metrics)
├── settings.py # Django settings
├── urls.py # Root URL configuration
└── celery.py # Celery configuration
```
---
## Models
### Base Models
#### AccountBaseModel
**File**: `auth/models.py`
**Purpose**: Base model for all account-isolated models.
**Fields**:
- `account`: ForeignKey to Account
- `created_at`: DateTimeField (auto_now_add)
- `updated_at`: DateTimeField (auto_now)
**Usage**: All models that need account isolation inherit from this.
#### SiteSectorBaseModel
**File**: `auth/models.py`
**Purpose**: Base model for models that belong to Site and Sector.
**Fields**:
- Inherits from `AccountBaseModel`
- `site`: ForeignKey to Site
- `sector`: ForeignKey to Sector
**Methods**:
- `save()`: Automatically sets `account` from `site.account` and validates sector belongs to site
**Usage**: Models like Keywords, Clusters, ContentIdeas, Tasks inherit from this.
### Auth Models
#### Account
**File**: `auth/models.py`
**Table**: `igny8_accounts`
**Fields**:
- `name`: CharField
- `slug`: SlugField (unique)
- `owner`: ForeignKey to User
- `stripe_customer_id`: CharField (optional)
- `plan`: ForeignKey to Plan
- `credits`: IntegerField (default: 0)
- `status`: CharField (choices: active, suspended, trial, cancelled)
**Methods**:
- `is_system_account()`: Returns True if account is system account (aws-admin, default-account, default)
#### User
**File**: `auth/models.py`
**Table**: `igny8_users`
**Fields**:
- Inherits from `AbstractUser`
- `email`: EmailField (unique, USERNAME_FIELD)
- `account`: ForeignKey to Account
- `role`: CharField (choices: developer, owner, admin, editor, viewer, system_bot)
**Methods**:
- `has_role(*roles)`: Check if user has any of the specified roles
- `is_owner_or_admin()`: Returns True if role is owner or admin
- `is_developer()`: Returns True if role is developer or is_superuser
- `is_admin_or_developer()`: Returns True if role is admin or developer
- `is_system_account_user()`: Returns True if user belongs to system account
- `get_accessible_sites()`: Returns QuerySet of sites user can access
#### Plan
**File**: `auth/models.py`
**Table**: `igny8_plans`
**Fields**:
- `name`: CharField
- `slug`: SlugField (unique)
- `price`: DecimalField
- `billing_cycle`: CharField (choices: monthly, annual)
- `features`: JSONField (array of feature strings)
- `max_users`: IntegerField
- `max_sites`: IntegerField
- `max_keywords`: IntegerField
- `max_clusters`: IntegerField
- `max_content_ideas`: IntegerField
- `daily_cluster_limit`: IntegerField
- `monthly_cluster_ai_credits`: IntegerField
- `daily_content_tasks`: IntegerField
- `daily_ai_requests`: IntegerField
- `monthly_word_count_limit`: IntegerField
- `monthly_content_ai_credits`: IntegerField
- `monthly_image_count`: IntegerField
- `daily_image_generation_limit`: IntegerField
- `monthly_image_ai_credits`: IntegerField
- `max_images_per_task`: IntegerField
- `image_model_choices`: JSONField
- `included_credits`: IntegerField
- `extra_credit_price`: DecimalField
- And more...
**Methods**:
- `clean()`: Validates plan limits
- `get_effective_credits_per_month()`: Returns included_credits or credits_per_month
#### Site
**File**: `auth/models.py`
**Table**: `igny8_sites`
**Fields**:
- Inherits from `AccountBaseModel`
- `name`: CharField
- `slug`: SlugField (unique per account)
- `domain`: URLField (optional)
- `industry`: ForeignKey to Industry (optional)
- `is_active`: BooleanField
- `status`: CharField (choices: active, inactive, suspended)
- `wp_url`: URLField (optional, WordPress integration)
- `wp_username`: CharField (optional)
- `wp_app_password`: CharField (optional)
**Methods**:
- `get_active_sectors_count()`: Get count of active sectors
- `can_add_sector()`: Check if site can add another sector (max 5)
#### Sector
**File**: `auth/models.py`
**Table**: `igny8_sectors`
**Fields**:
- Inherits from `AccountBaseModel`
- `site`: ForeignKey to Site
- `industry_sector`: ForeignKey to IndustrySector (optional, template reference)
- `name`: CharField
- `slug`: SlugField (unique per site)
- `is_active`: BooleanField
- `status`: CharField (choices: active, inactive)
### Planner Models
#### Keywords
**File**: `modules/planner/models.py`
**Table**: `igny8_keywords`
**Fields**:
- Inherits from `SiteSectorBaseModel`
- `keyword`: CharField
- `volume`: IntegerField
- `difficulty`: IntegerField
- `intent`: CharField (choices: informational, navigational, commercial, transactional)
- `cluster`: ForeignKey to Clusters (optional)
- `status`: CharField (choices: active, pending, archived)
#### Clusters
**File**: `modules/planner/models.py`
**Table**: `igny8_clusters`
**Fields**:
- Inherits from `SiteSectorBaseModel`
- `name`: CharField (unique)
- `description`: TextField
- `keywords_count`: IntegerField
- `volume`: IntegerField
- `mapped_pages`: IntegerField
- `status`: CharField
#### ContentIdeas
**File**: `modules/planner/models.py`
**Table**: `igny8_content_ideas`
**Fields**:
- Inherits from `SiteSectorBaseModel`
- `idea_title`: CharField
- `description`: TextField
- `content_structure`: CharField (choices: cluster_hub, landing_page, pillar_page, supporting_page)
- `content_type`: CharField (choices: blog_post, article, guide, tutorial)
- `target_keywords`: CharField (comma-separated, legacy)
- `keyword_objects`: ManyToManyField to Keywords
- `keyword_cluster`: ForeignKey to Clusters
- `status`: CharField (choices: new, scheduled, published)
- `estimated_word_count`: IntegerField
### Writer Models
#### Tasks
**File**: `modules/writer/models.py`
**Table**: `igny8_tasks`
**Fields**:
- Inherits from `SiteSectorBaseModel`
- `title`: CharField
- `description`: TextField
- `keywords`: CharField (comma-separated, legacy)
- `keyword_objects`: ManyToManyField to Keywords
- `cluster`: ForeignKey to Clusters
- `idea`: ForeignKey to ContentIdeas
- `content_structure`: CharField
- `content_type`: CharField
- `status`: CharField (choices: queued, in_progress, draft, review, published, completed)
- `content`: TextField (generated content)
- `word_count`: IntegerField
- `meta_title`: CharField
- `meta_description`: TextField
- `assigned_post_id`: IntegerField (WordPress post ID)
- `post_url`: URLField
#### Content
**File**: `modules/writer/models.py`
**Table**: `igny8_content`
**Fields**:
- Inherits from `SiteSectorBaseModel`
- `task`: OneToOneField to Tasks
- `html_content`: TextField
- `word_count`: IntegerField
- `metadata`: JSONField
**Methods**:
- `save()`: Automatically sets account, site, sector from task
#### Images
**File**: `modules/writer/models.py`
**Table**: `igny8_images`
**Fields**:
- Inherits from `SiteSectorBaseModel`
- `task`: ForeignKey to Tasks
- `image_type`: CharField (choices: featured, desktop, mobile, in_article)
- `image_url`: URLField
- `image_path`: CharField (local path)
- `prompt`: TextField
- `status`: CharField
- `position`: IntegerField
**Methods**:
- `save()`: Automatically sets account, site, sector from task
### System Models
#### AIPrompt
**File**: `modules/system/models.py`
**Table**: `igny8_ai_prompts`
**Fields**:
- Inherits from `AccountBaseModel`
- `prompt_type`: CharField (choices: clustering, ideas, content_generation, image_prompt_extraction, image_prompt_template, negative_prompt)
- `prompt_value`: TextField
- `default_prompt`: TextField
- `is_active`: BooleanField
**Unique Constraint**: `(account, prompt_type)`
#### IntegrationSettings
**File**: `modules/system/models.py`
**Table**: `igny8_integration_settings`
**Fields**:
- Inherits from `AccountBaseModel`
- `integration_type`: CharField (choices: openai, runware, gsc, image_generation)
- `config`: JSONField (API keys, settings, etc.)
- `is_active`: BooleanField
**Unique Constraint**: `(account, integration_type)`
#### AuthorProfile
**File**: `modules/system/models.py`
**Table**: `igny8_author_profiles`
**Fields**:
- Inherits from `AccountBaseModel`
- `name`: CharField
- `description`: TextField
- `tone`: CharField
- `language`: CharField
- `structure_template`: JSONField
- `is_active`: BooleanField
#### Strategy
**File**: `modules/system/models.py`
**Table**: `igny8_strategies`
**Fields**:
- Inherits from `AccountBaseModel`
- `name`: CharField
- `description`: TextField
- `sector`: ForeignKey to Sector (optional)
- `prompt_types`: JSONField
- `section_logic`: JSONField
- `is_active`: BooleanField
---
## ViewSets
### Base ViewSets
#### AccountModelViewSet
**File**: `api/base.py`
**Purpose**: Base ViewSet with automatic account filtering.
**Methods**:
- `get_queryset()`: Filters queryset by `request.account` (with admin/developer override)
- `perform_create()`: Sets account on created objects
- `get_serializer_context()`: Adds account to serializer context
**Access Control**:
- Admin/Developer users: Bypass account filtering
- System account users: Bypass account filtering
- Regular users: Only see data from their account
#### SiteSectorModelViewSet
**File**: `api/base.py`
**Purpose**: Base ViewSet with site/sector filtering and access control.
**Inherits**: `AccountModelViewSet`
**Methods**:
- `get_queryset()`: Filters by account, accessible sites, and optional site_id/sector_id
- `perform_create()`: Validates site access and sector-site relationship
- `get_serializer_context()`: Adds accessible sites and sectors to context
**Access Control**:
- Developers: All active sites
- System account users: All active sites
- Owners/Admins: All sites in their account
- Editors/Viewers: Only sites granted via `SiteUserAccess`
### Planner ViewSets
#### KeywordViewSet
**File**: `modules/planner/views.py`
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List keywords with filtering
- `create()`: Create keyword
- `retrieve()`: Get keyword details
- `update()`: Update keyword
- `destroy()`: Delete keyword
- `auto_cluster()`: Auto-cluster keywords using AI
- `bulk_delete()`: Bulk delete keywords
- `bulk_update_status()`: Bulk update keyword status
- `export_csv()`: Export keywords to CSV
- `import_csv()`: Import keywords from CSV
**Filtering**:
- Search: `keyword` field
- Filters: `status`, `cluster_id`, `intent`
- Custom: `difficulty_min`, `difficulty_max`, `volume_min`, `volume_max`
- Ordering: `created_at`, `volume`, `difficulty`
#### ClusterViewSet
**File**: `modules/planner/views.py`
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List clusters
- `create()`: Create cluster
- `retrieve()`: Get cluster details
- `update()`: Update cluster
- `destroy()`: Delete cluster
- `auto_generate_ideas()`: Auto-generate content ideas for clusters
#### ContentIdeasViewSet
**File**: `modules/planner/views.py`
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List content ideas
- `create()`: Create content idea
- `retrieve()`: Get content idea details
- `update()`: Update content idea
- `destroy()`: Delete content idea
### Writer ViewSets
#### TasksViewSet
**File**: `modules/writer/views.py`
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List tasks
- `create()`: Create task
- `retrieve()`: Get task details
- `update()`: Update task
- `destroy()`: Delete task
- `auto_generate_content()`: Auto-generate content for tasks
- `auto_generate_images()`: Auto-generate images for tasks
- `bulk_delete()`: Bulk delete tasks
- `bulk_update()`: Bulk update task status
**Filtering**:
- Search: `title`, `keywords`
- Filters: `status`, `cluster_id`, `content_type`, `content_structure`
- Ordering: `title`, `created_at`, `word_count`, `status`
### System ViewSets
#### IntegrationSettingsViewSet
**File**: `modules/system/integration_views.py`
**Inherits**: `viewsets.ViewSet`
**Actions**:
- `list()`: List integrations
- `retrieve()`: Get integration settings
- `update()`: Save integration settings
- `save_post()`: Save integration settings (POST)
- `test_connection()`: Test API connection
- `test_openai()`: Test OpenAI connection
- `test_runware()`: Test Runware connection
- `generate_image()`: Test image generation
- `task_progress()`: Get Celery task progress
#### AIPromptViewSet
**File**: `modules/system/views.py`
**Inherits**: `AccountModelViewSet`
**Actions**:
- `list()`: List prompts
- `create()`: Create prompt
- `retrieve()`: Get prompt details
- `update()`: Update prompt
- `destroy()`: Delete prompt
- `reset_to_default()`: Reset prompt to default value
#### AuthorProfileViewSet
**File**: `modules/system/views.py`
**Inherits**: `AccountModelViewSet`
**Actions**:
- `list()`: List author profiles
- `create()`: Create author profile
- `retrieve()`: Get author profile details
- `update()`: Update author profile
- `destroy()`: Delete author profile
---
## Serializers
### Planner Serializers
#### KeywordSerializer
**File**: `modules/planner/serializers.py`
**Fields**: All Keyword model fields
**Validation**: Validates keyword uniqueness, cluster belongs to same sector
#### ClusterSerializer
**File**: `modules/planner/cluster_serializers.py`
**Fields**: All Cluster model fields
**Read-Only Fields**: `keywords_count`, `volume` (calculated)
#### ContentIdeasSerializer
**File**: `modules/planner/serializers.py`
**Fields**: All ContentIdeas model fields
### Writer Serializers
#### TasksSerializer
**File**: `modules/writer/serializers.py`
**Fields**: All Tasks model fields
#### ContentSerializer
**File**: `modules/writer/serializers.py`
**Fields**: All Content model fields
#### ImagesSerializer
**File**: `modules/writer/serializers.py`
**Fields**: All Images model fields
### System Serializers
#### AIPromptSerializer
**File**: `modules/system/serializers.py`
**Fields**: All AIPrompt model fields
#### IntegrationSettingsSerializer
**File**: `modules/system/serializers.py`
**Fields**: All IntegrationSettings model fields
---
## Celery Tasks
### Planner Tasks
#### auto_cluster_keywords_task
**File**: `modules/planner/tasks.py`
**Purpose**: Auto-cluster keywords using AI.
**Parameters**:
- `keyword_ids`: List of keyword IDs
- `account_id`: Account ID
- `site_id`: Site ID
- `sector_id`: Sector ID
**Progress Tracking**: Updates progress with request_steps and response_steps
**Calls**: `_auto_cluster_keywords_core()`
#### auto_generate_ideas_task
**File**: `modules/planner/tasks.py`
**Purpose**: Auto-generate content ideas for clusters.
**Parameters**:
- `cluster_ids`: List of cluster IDs
- `account_id`: Account ID
**Progress Tracking**: Updates progress for each cluster
**Calls**: `_generate_single_idea_core()` for each cluster
### Writer Tasks
#### auto_generate_content_task
**File**: `modules/writer/tasks.py`
**Purpose**: Auto-generate content for tasks.
**Parameters**:
- `task_ids`: List of task IDs
- `account_id`: Account ID
**Progress Tracking**: Updates progress for each task
**Calls**: `AIProcessor.generate_content()`
#### auto_generate_images_task
**File**: `modules/writer/tasks.py`
**Purpose**: Auto-generate images for tasks.
**Parameters**:
- `task_ids`: List of task IDs
- `account_id`: Account ID
**Progress Tracking**: Updates progress for each task
**Calls**: `AIProcessor.extract_image_prompts()` and `AIProcessor.generate_image()`
---
## API Endpoints
### Base URL
`/api/v1/`
### Planner Endpoints
- `GET /api/v1/planner/keywords/` - List keywords
- `POST /api/v1/planner/keywords/` - Create keyword
- `GET /api/v1/planner/keywords/{id}/` - Get keyword
- `PUT /api/v1/planner/keywords/{id}/` - Update keyword
- `DELETE /api/v1/planner/keywords/{id}/` - Delete keyword
- `POST /api/v1/planner/keywords/auto_cluster/` - Auto-cluster keywords
- `POST /api/v1/planner/keywords/bulk_delete/` - Bulk delete keywords
- `POST /api/v1/planner/keywords/bulk_update_status/` - Bulk update status
- `GET /api/v1/planner/keywords/export_csv/` - Export keywords
- `POST /api/v1/planner/keywords/import_csv/` - Import keywords
- `GET /api/v1/planner/clusters/` - List clusters
- `POST /api/v1/planner/clusters/auto_generate_ideas/` - Auto-generate ideas
- `GET /api/v1/planner/ideas/` - List content ideas
### Writer Endpoints
- `GET /api/v1/writer/tasks/` - List tasks
- `POST /api/v1/writer/tasks/auto_generate_content/` - Auto-generate content
- `POST /api/v1/writer/tasks/auto_generate_images/` - Auto-generate images
### System Endpoints
- `GET /api/v1/system/settings/integrations/{pk}/` - Get integration settings
- `PUT /api/v1/system/settings/integrations/{pk}/` - Save integration settings
- `POST /api/v1/system/settings/integrations/{pk}/test_openai/` - Test OpenAI
- `POST /api/v1/system/settings/integrations/{pk}/test_runware/` - Test Runware
- `POST /api/v1/system/settings/integrations/{pk}/generate_image/` - Test image generation
- `GET /api/v1/system/settings/task_progress/{task_id}/` - Get task progress
---
## Base Classes
### AccountModelViewSet
**File**: `api/base.py`
**Purpose**: Base ViewSet with automatic account filtering.
**Features**:
- Automatic account filtering
- Admin/Developer override
- Account context in serializers
### SiteSectorModelViewSet
**File**: `api/base.py`
**Purpose**: Base ViewSet with site/sector filtering.
**Features**:
- Account filtering (inherited)
- Site access control
- Sector validation
- Accessible sites/sectors in serializer context
---
## Middleware
### AccountContextMiddleware
**File**: `middleware/account.py`
**Purpose**: Sets `request.account` from JWT token.
**Functionality**:
- Extracts account ID from JWT token
- Loads Account object
- Sets `request.account`
### ResourceTrackerMiddleware
**File**: `middleware/resource_tracker.py`
**Purpose**: Tracks API request metrics.
**Functionality**:
- Tracks CPU, memory, I/O usage
- Stores metrics in cache
- Provides metrics endpoint
---
## Utilities
### AIProcessor
**File**: `utils/ai_processor.py`
**Purpose**: Unified AI interface for all AI operations.
**Methods**:
- `cluster_keywords()`: Cluster keywords using AI
- `generate_ideas()`: Generate content ideas
- `generate_content()`: Generate text content
- `extract_image_prompts()`: Extract image prompts from content
- `generate_image()`: Generate images using OpenAI DALL-E or Runware
**See**: AI Functions documentation for complete details
### Content Normalizer
**File**: `utils/content_normalizer.py`
**Purpose**: Content processing utilities.
**Functions**:
- `_extract_body_content()`: Extract body content from HTML
---
## Modules
### Planner Module
**Purpose**: Keyword management and content planning.
**Models**: Keywords, Clusters, ContentIdeas
**ViewSets**: KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
**Tasks**: auto_cluster_keywords_task, auto_generate_ideas_task
### Writer Module
**Purpose**: Content generation and management.
**Models**: Tasks, Content, Images
**ViewSets**: TasksViewSet
**Tasks**: auto_generate_content_task, auto_generate_images_task
### System Module
**Purpose**: System settings, prompts, and integrations.
**Models**: AIPrompt, IntegrationSettings, AuthorProfile, Strategy
**ViewSets**: AIPromptViewSet, AuthorProfileViewSet, IntegrationSettingsViewSet
**Utilities**: Default prompts, prompt loading
### Billing Module
**Purpose**: Credits, transactions, and usage tracking.
**Models**: CreditTransaction, UsageLog
**ViewSets**: CreditTransactionViewSet, UsageLogViewSet
**Services**: CreditService
---
## Summary
The IGNY8 backend is built on:
1. **Django + DRF**: Robust web framework with RESTful API
2. **Multi-Tenancy**: Complete account isolation with automatic filtering
3. **Modular Architecture**: Clear module boundaries with shared utilities
4. **Celery Integration**: Asynchronous task processing for long-running operations
5. **Base ViewSets**: Consistent access control and filtering
6. **AI Integration**: Unified AIProcessor for all AI operations
7. **Progress Tracking**: Real-time progress updates for Celery tasks
8. **Account/Site/Sector Hierarchy**: Hierarchical data organization
This architecture ensures scalability, maintainability, and extensibility while providing a robust API for the frontend.

666
docs/ActiveDocs/04-AI-FUNCTIONS.md Normal file
View File

@@ -0,0 +1,666 @@
# IGNY8 AI Functions Documentation
**Version:** 1.0
**Last Updated:** 2025-01-XX
**Purpose:** Complete documentation of all AI functions in the IGNY8 system, their workflows, dependencies, and integration points.
---
## Table of Contents
1. [AI Functions Overview](#ai-functions-overview)
2. [AIProcessor Class](#aiprocessor-class)
3. [Planner Module AI Functions](#planner-module-ai-functions)
4. [Writer Module AI Functions](#writer-module-ai-functions)
5. [System Module AI Functions](#system-module-ai-functions)
6. [Progress Tracking](#progress-tracking)
7. [Error Handling](#error-handling)
8. [Dependencies](#dependencies)
---
## AI Functions Overview
The IGNY8 system includes **8 primary AI functions** across 3 modules:
1. **Planner Module (2 functions):**
- Auto Cluster Keywords (`auto_cluster`)
- Auto Generate Ideas (`auto_generate_ideas`)
2. **Writer Module (3 functions):**
- Auto Generate Content (`auto_generate_content`)
- Auto Generate Images (`auto_generate_images`)
- Extract Image Prompts (`extract_image_prompts`)
3. **System Module (3 functions):**
- Test OpenAI Integration (`test_openai`)
- Test Runware Integration (`test_runware`)
- Generate Image (Test) (`generate_image`)
All AI functions use the unified `AIProcessor` class as the core interface for AI operations.
---
## AIProcessor Class
**Location**: `backend/igny8_core/utils/ai_processor.py`
**Purpose**: Unified AI interface for all AI operations.
### Initialization
**Method**: `AIProcessor.__init__()`
**Parameters**:
- `account` (optional): Account object for loading account-specific settings
**Dependencies**:
- `IntegrationSettings` model (for API keys and model configuration)
- Django settings (fallback for API keys)
**Initializes**:
- `self.openai_api_key`: Loaded from IntegrationSettings or Django settings
- `self.runware_api_key`: Loaded from IntegrationSettings or Django settings
- `self.default_model`: Loaded from IntegrationSettings or Django settings
- `self.model_rates`: Model pricing rates (per 1M tokens)
- `self.image_model_rates`: Image model pricing rates (per image)
### Core Methods
#### cluster_keywords()
**Purpose**: Cluster keywords using AI-based semantic similarity.
**Parameters**:
- `keywords`: List of keyword dictionaries
- `sector_name`: Optional sector name for context
- `account`: Optional account object
- `response_steps`: Optional list for step tracking
- `progress_callback`: Optional callback function
**Dependencies**:
- `AIPrompt` model (for clustering prompt template)
- `get_default_prompt()` utility function
- `_call_openai()` method
- `_extract_json_from_response()` method
**Returns**: Dictionary with `clusters` array and `response_steps`
#### generate_ideas()
**Purpose**: Generate content ideas for clusters using AI.
**Parameters**:
- `clusters`: List of cluster dictionaries
- `account`: Optional account object
**Dependencies**:
- `AIPrompt` model (for ideas prompt template)
- `get_default_prompt()` utility function
- `_call_openai()` method
- `_extract_json_from_response()` method
**Returns**: Dictionary with `ideas` array
#### generate_content()
**Purpose**: Generate text content using OpenAI GPT models.
**Parameters**:
- `prompt`: The prompt text
- `model`: Optional model name (defaults to self.default_model)
- `max_tokens`: Maximum tokens to generate
- `temperature`: Sampling temperature
**Dependencies**:
- `_call_openai()` method
**Returns**: Dictionary with `content`, `tokens_used`, `model`, `cost`, `error`
#### extract_image_prompts()
**Purpose**: Extract image prompts from content using AI.
**Parameters**:
- `content`: Article content text
- `title`: Article title
- `max_images`: Maximum number of images to extract prompts for
- `account`: Optional account object
**Dependencies**:
- `AIPrompt` model (for image prompt extraction template)
- `get_default_prompt()` utility function
- `_call_openai()` method
- `_extract_json_from_response()` method
**Returns**: Dictionary with `featured_prompt` and `in_article_prompts` array
#### generate_image()
**Purpose**: Generate images using OpenAI DALL-E or Runware.
**Parameters**:
- `prompt`: Image generation prompt
- `model`: Image model name (dall-e-3, dall-e-2, or runware model)
- `size`: Image size (e.g., "1024x1024")
- `n`: Number of images to generate
- `account`: Optional account object
**Dependencies**:
- `IntegrationSettings` model (for API keys)
- OpenAI API or Runware API
**Returns**: Dictionary with `image_url`, `revised_prompt`, `cost`, `provider`, `model`
### Internal Methods
#### _call_openai()
**Purpose**: Make HTTP request to OpenAI API.
**Parameters**:
- `prompt`: The prompt text
- `model`: Model name
- `max_tokens`: Maximum tokens
- `temperature`: Sampling temperature
- `response_format`: Optional JSON response format
- `response_steps`: Optional list for step tracking
**Dependencies**:
- `requests` library
- OpenAI API endpoint
**Returns**: Dictionary with `content`, `input_tokens`, `output_tokens`, `total_tokens`, `model`, `cost`, `error`, `api_id`
#### _extract_json_from_response()
**Purpose**: Extract JSON from OpenAI response (handles various formats).
**Parameters**:
- `response_text`: Raw response text from OpenAI
**Dependencies**:
- `json` library
- `re` library (for regex patterns)
**Returns**: Parsed JSON dictionary or None
#### _get_api_key()
**Purpose**: Get API key from IntegrationSettings or Django settings.
**Parameters**:
- `integration_type`: Type of integration ('openai' or 'runware')
- `account`: Optional account object
**Dependencies**:
- `IntegrationSettings` model
- Django settings
**Returns**: API key string or None
#### _get_model()
**Purpose**: Get default model from IntegrationSettings or Django settings.
**Parameters**:
- `integration_type`: Type of integration ('openai')
- `account`: Optional account object
**Dependencies**:
- `IntegrationSettings` model
- Django settings
**Returns**: Model name string or default
#### get_prompt()
**Purpose**: Get prompt template from AIPrompt model or default.
**Parameters**:
- `prompt_type`: Type of prompt ('clustering', 'ideas', 'content_generation', etc.)
- `account`: Optional account object
**Dependencies**:
- `AIPrompt` model
- `get_default_prompt()` utility function
**Returns**: Prompt template string
---
## Planner Module AI Functions
### 1. Auto Cluster Keywords
**Function Name**: `auto_cluster`
**API Endpoint**: `POST /v1/planner/keywords/auto_cluster/`
**ViewSet Action**: `KeywordViewSet.auto_cluster()`
**Celery Task**: `auto_cluster_keywords_task()`
**Core Function**: `_auto_cluster_keywords_core()`
**AIProcessor Method**: `AIProcessor.cluster_keywords()`
**Purpose**: Automatically group related keywords into semantic clusters using AI.
#### Complete Workflow (16 Steps)
**Request Phase (Steps 1-7)**:
1. **API Endpoint Validation**: Validate keyword IDs array
2. **Account/Site Extraction**: Extract account and site from request
3. **Task Queuing**: Queue Celery task
4. **Keyword Loading & Validation**: Load keywords from database
5. **Relationship Validation**: Validate account, site, sector relationships
6. **AIProcessor Creation**: Create AIProcessor instance
7. **AI Call Preparation**: Format keywords for AI processing
**AI Processing Phase (Steps 8-12)**:
8. **HTTP Response Validation**: Validate OpenAI API response
9. **Content Extraction**: Extract content from response
10. **Cost Calculation**: Calculate API cost
11. **JSON Extraction & Parsing**: Extract and parse JSON from response
12. **Cluster Data Validation**: Validate cluster data structure
**Database Phase (Steps 13-16)**:
13. **Database Transaction Start**: Begin atomic transaction
14. **Cluster Creation/Update**: Create or update clusters
15. **Keyword Matching & Assignment**: Match keywords to clusters
16. **Metrics Recalculation & Commit**: Recalculate cluster metrics and commit
#### Dependencies
**Models**:
- `Keywords` model
- `Clusters` model
- `Account` model
- `Site` model
- `Sector` model
- `IntegrationSettings` model (for OpenAI API key)
- `AIPrompt` model (for clustering prompt template)
- `Plan` model (for plan limits)
**Functions**:
- `KeywordViewSet.auto_cluster()` (API endpoint)
- `auto_cluster_keywords_task()` (Celery task)
- `_auto_cluster_keywords_core()` (core function)
- `AIProcessor.cluster_keywords()` (AI processing)
**Files**:
- `backend/igny8_core/modules/planner/views.py`
- `backend/igny8_core/modules/planner/tasks.py`
- `backend/igny8_core/utils/ai_processor.py`
### 2. Auto Generate Ideas
**Function Name**: `auto_generate_ideas`
**API Endpoint**: `POST /v1/planner/clusters/auto_generate_ideas/`
**ViewSet Action**: `ClusterViewSet.auto_generate_ideas()`
**Celery Task**: `auto_generate_ideas_task()`
**Core Function**: `_generate_single_idea_core()`
**AIProcessor Method**: `AIProcessor.generate_ideas()`
**Purpose**: Generate content ideas for keyword clusters using AI.
#### Workflow
1. **API Endpoint**: Validate cluster IDs, extract account
2. **Task Queuing**: Queue Celery task
3. **Cluster Loading**: Load clusters from database
4. **AI Processing**: Call `AIProcessor.generate_ideas()`
5. **Idea Creation**: Create ContentIdeas records
#### Dependencies
**Models**:
- `Clusters` model
- `ContentIdeas` model
- `Keywords` model
- `Account` model
- `Site` model
- `Sector` model
- `IntegrationSettings` model (for OpenAI API key)
- `AIPrompt` model (for ideas prompt template)
- `Plan` model (for plan limits)
**Functions**:
- `ClusterViewSet.auto_generate_ideas()` (API endpoint)
- `auto_generate_ideas_task()` (Celery task)
- `_generate_single_idea_core()` (core function)
- `AIProcessor.generate_ideas()` (AI processing)
**Files**:
- `backend/igny8_core/modules/planner/views.py`
- `backend/igny8_core/modules/planner/tasks.py`
- `backend/igny8_core/utils/ai_processor.py`
---
## Writer Module AI Functions
### 3. Auto Generate Content
**Function Name**: `auto_generate_content`
**API Endpoint**: `POST /v1/writer/tasks/auto_generate_content/`
**ViewSet Action**: `TasksViewSet.auto_generate_content()`
**Celery Task**: `auto_generate_content_task()`
**AIProcessor Method**: `AIProcessor.generate_content()`
**Purpose**: Generate complete blog post/article content using AI.
#### Workflow
1. **API Endpoint**: Validate task IDs, extract account
2. **Task Queuing**: Queue Celery task
3. **Task Loading**: Load tasks with relationships
4. **Content Generation**: Call `AIProcessor.generate_content()` for each task
5. **Content Saving**: Save generated content to Content model
#### Dependencies
**Models**:
- `Tasks` model
- `Content` model
- `ContentIdeas` model
- `Clusters` model
- `Keywords` model
- `Account` model
- `Site` model
- `Sector` model
- `IntegrationSettings` model (for OpenAI API key)
- `AIPrompt` model (for content generation prompt template)
- `AuthorProfile` model (for writing style)
- `Plan` model (for plan limits)
**Functions**:
- `TasksViewSet.auto_generate_content()` (API endpoint)
- `auto_generate_content_task()` (Celery task)
- `AIProcessor.generate_content()` (AI processing)
**Files**:
- `backend/igny8_core/modules/writer/views.py`
- `backend/igny8_core/modules/writer/tasks.py`
- `backend/igny8_core/utils/ai_processor.py`
### 4. Auto Generate Images
**Function Name**: `auto_generate_images`
**API Endpoint**: `POST /v1/writer/tasks/auto_generate_images/`
**ViewSet Action**: `TasksViewSet.auto_generate_images()`
**Celery Task**: `auto_generate_images_task()`
**AIProcessor Methods**: `AIProcessor.extract_image_prompts()`, `AIProcessor.generate_image()`
**Purpose**: Generate images for content tasks using AI (OpenAI DALL-E or Runware).
#### Workflow
1. **API Endpoint**: Validate task IDs, extract account
2. **Task Queuing**: Queue Celery task
3. **Task Loading**: Load tasks with content
4. **Image Prompt Extraction**: Call `AIProcessor.extract_image_prompts()`
5. **Image Generation**: Call `AIProcessor.generate_image()` for each prompt
6. **Image Saving**: Save generated images to Images model
#### Dependencies
**Models**:
- `Tasks` model
- `Images` model
- `Content` model
- `Account` model
- `IntegrationSettings` model (for OpenAI/Runware API keys)
- `AIPrompt` model (for image prompt templates)
- `Plan` model (for plan limits and image model choices)
**Functions**:
- `TasksViewSet.auto_generate_images()` (API endpoint)
- `auto_generate_images_task()` (Celery task)
- `AIProcessor.extract_image_prompts()` (prompt extraction)
- `AIProcessor.generate_image()` (image generation)
**Files**:
- `backend/igny8_core/modules/writer/views.py`
- `backend/igny8_core/modules/writer/tasks.py`
- `backend/igny8_core/utils/ai_processor.py`
### 5. Extract Image Prompts
**Function Name**: `extract_image_prompts`
**AIProcessor Method**: `AIProcessor.extract_image_prompts()`
**Purpose**: Extract image prompts from generated content for image generation.
**Note**: This is typically called internally by `auto_generate_images`, but can be called directly.
#### Dependencies
**Models**:
- `Content` model
- `AIPrompt` model (for image prompt extraction template)
- `IntegrationSettings` model (for OpenAI API key)
**Functions**:
- `AIProcessor.extract_image_prompts()` (AI processing)
**Files**:
- `backend/igny8_core/utils/ai_processor.py`
---
## System Module AI Functions
### 6. Test OpenAI Integration
**Function Name**: `test_openai`
**API Endpoint**: `POST /v1/system/settings/integrations/{id}/test_openai/`
**ViewSet Action**: `IntegrationSettingsViewSet.test_openai()`
**Purpose**: Test OpenAI API connection and validate API key.
#### Workflow
1. **Get API Key**: From request or saved IntegrationSettings
2. **Test Connection**: Call OpenAI API (`/v1/models` or `/v1/chat/completions`)
3. **Return Result**: Success or error message
#### Dependencies
**Models**:
- `IntegrationSettings` model
- `Account` model
**Functions**:
- `IntegrationSettingsViewSet.test_openai()` (API endpoint)
- `AIProcessor` class (for API key loading)
**Files**:
- `backend/igny8_core/modules/system/integration_views.py`
- `backend/igny8_core/utils/ai_processor.py`
### 7. Test Runware Integration
**Function Name**: `test_runware`
**API Endpoint**: `POST /v1/system/settings/integrations/{id}/test_runware/`
**ViewSet Action**: `IntegrationSettingsViewSet.test_runware()`
**Purpose**: Test Runware API connection and validate API key.
#### Workflow
1. **Get API Key**: From request or saved IntegrationSettings
2. **Test Connection**: Call Runware API (`/v1`)
3. **Return Result**: Success or error message
#### Dependencies
**Models**:
- `IntegrationSettings` model
- `Account` model
**Functions**:
- `IntegrationSettingsViewSet.test_runware()` (API endpoint)
- `AIProcessor` class (for API key loading)
**Files**:
- `backend/igny8_core/modules/system/integration_views.py`
- `backend/igny8_core/utils/ai_processor.py`
### 8. Generate Image (Test)
**Function Name**: `generate_image`
**API Endpoint**: `POST /v1/system/settings/integrations/{id}/generate_image/`
**ViewSet Action**: `IntegrationSettingsViewSet.generate_image()`
**Purpose**: Test image generation with OpenAI DALL-E or Runware.
#### Workflow
1. **Get Configuration**: From request or saved IntegrationSettings
2. **Generate Image**: Call `AIProcessor.generate_image()`
3. **Return Result**: Image URL or error message
#### Dependencies
**Models**:
- `IntegrationSettings` model
- `Account` model
**Functions**:
- `IntegrationSettingsViewSet.generate_image()` (API endpoint)
- `AIProcessor.generate_image()` (image generation)
**Files**:
- `backend/igny8_core/modules/system/integration_views.py`
- `backend/igny8_core/utils/ai_processor.py`
---
## Progress Tracking
### Task Progress Endpoint
**Endpoint**: `GET /v1/system/settings/task_progress/{task_id}/`
**ViewSet Action**: `IntegrationSettingsViewSet.task_progress()`
**Purpose**: Get Celery task status and progress.
**Response Format**:
```json
{
"state": "PROGRESS",
"meta": {
"current": 2,
"total": 10,
"percentage": 20,
"message": "Processing...",
"request_steps": [...],
"response_steps": [...]
}
}
```
**States**:
- `PENDING`: Task is queued but not started
- `PROGRESS`: Task is in progress
- `SUCCESS`: Task completed successfully
- `FAILURE`: Task failed with error
### Frontend Integration
**Hook**: `useProgressModal`
**Functionality**:
- Polls task progress endpoint every 2 seconds
- Displays progress percentage and message
- Shows step-by-step logs (request_steps and response_steps)
- Auto-closes on completion
**Store**: `aiRequestLogsStore`
**Functionality**:
- Stores AI request/response logs
- Tracks step-by-step execution
- Displays logs in UI
---
## Error Handling
### Error Types
1. **API Errors**: OpenAI/Runware API errors
2. **Validation Errors**: Input validation failures
3. **Database Errors**: Database operation failures
4. **Connection Errors**: Celery/Redis connection issues
### Error Propagation
1. **API Endpoint**: Catches errors, returns error response with request_steps
2. **Celery Task**: Catches errors, updates task state to FAILURE with error details
3. **Core Function**: Catches errors, logs to request_steps/response_steps
4. **AIProcessor**: Catches errors, returns error in result dictionary
### Error Logging
- All errors logged to `request_steps` or `response_steps`
- Error details include: error type, message, function name, step number
- Frontend displays errors in AI Request/Response Logs section
---
## Dependencies
### Database Models
- `Account`: Account isolation
- `Site`, `Sector`: Site/sector hierarchy
- `Keywords`, `Clusters`, `ContentIdeas`: Planner data
- `Tasks`, `Content`, `Images`: Writer data
- `IntegrationSettings`: API keys and configuration
- `AIPrompt`: Prompt templates
- `Plan`: Plan limits
### External Services
- **OpenAI API**: GPT models for text generation
- **OpenAI DALL-E**: Image generation
- **Runware API**: Alternative image generation
### Utilities
- `get_default_prompt()`: Default prompt templates
- `AIProcessor`: Unified AI interface
- Celery: Asynchronous task processing
- Redis: Celery broker
---
## Summary
The IGNY8 AI functions system provides:
1. **Unified Interface**: Single AIProcessor class for all AI operations
2. **Account Isolation**: All AI functions respect account boundaries
3. **Progress Tracking**: Real-time progress updates with step-by-step logs
4. **Error Handling**: Comprehensive error logging and propagation
5. **Flexible Configuration**: Account-specific API keys and prompts
6. **Asynchronous Processing**: Celery tasks for long-running operations
7. **Multiple Providers**: Support for OpenAI and Runware
8. **Complete Workflows**: End-to-end workflows from API to database
All AI functions integrate seamlessly with the account/site/user/plan architecture and provide detailed logging for debugging and monitoring.

739
docs/ActiveDocs/05-ACCOUNT-USER-PLAN.md Normal file
View File

@@ -0,0 +1,739 @@
# IGNY8 Account, User, Plan, Sites, Credits Documentation
**Version:** 1.0
**Last Updated:** 2025-01-XX
**Purpose:** Complete documentation of multi-tenancy architecture, access control, plans, credits, and related systems.
---
## Table of Contents
1. [Overview](#overview)
2. [Account Model](#account-model)
3. [User Model](#user-model)
4. [Plan Model](#plan-model)
5. [Subscription Model](#subscription-model)
6. [Site Model](#site-model)
7. [Sector Model](#sector-model)
8. [Industry & IndustrySector Models](#industry--industrysector-models)
9. [Access Control](#access-control)
10. [Credits System](#credits-system)
11. [Plan Limits](#plan-limits)
12. [Multi-Tenancy Architecture](#multi-tenancy-architecture)
---
## Overview
The IGNY8 platform uses a multi-account architecture with the following core entities:
- **Account**: Top-level organization/workspace
- **User**: Individual user accounts with roles
- **Plan**: Subscription plan templates with limits
- **Subscription**: Active subscription linking Account to Plan
- **Site**: Workspace within an Account (1-N relationship)
- **Sector**: Content category within a Site (1-5 per site)
- **Industry**: Global industry templates
- **IndustrySector**: Industry sector templates
### Key Relationships
```
Account (1) ──< (N) User
Account (1) ──< (1) Subscription ──> (1) Plan
Account (1) ──< (N) Site
Site (1) ──< (1-5) Sector
Industry (1) ──< (N) IndustrySector
Site (1) ──> (1) Industry (optional)
Sector (1) ──> (1) IndustrySector (optional, template reference)
```
---
## Account Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_accounts`
### Fields
- `id`: Primary key
- `name`: CharField - Account name
- `slug`: SlugField - Unique slug identifier
- `owner`: ForeignKey to User - Account owner
- `stripe_customer_id`: CharField - Stripe customer ID (optional)
- `plan`: ForeignKey to Plan - Current subscription plan
- `credits`: IntegerField - Current credit balance (default: 0)
- `status`: CharField - Account status (choices: active, suspended, trial, cancelled)
- `created_at`: DateTimeField - Creation timestamp
- `updated_at`: DateTimeField - Last update timestamp
### Status Values
- `active`: Normal operation
- `suspended`: Access temporarily revoked
- `trial`: Limited trial access
- `cancelled`: Account terminated
### Methods
#### is_system_account()
**Returns**: `True` if account is a system account (aws-admin, default-account, default)
**Purpose**: System accounts bypass all filtering restrictions
### Relationships
- **One-to-Many**: Users (via `user.account`)
- **One-to-One**: Subscription (via `account.subscription`)
- **One-to-Many**: Sites (via `site.account`)
---
## User Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_users`
**Inherits**: `AbstractUser` (Django's built-in user model)
### Fields
- `id`: Primary key (inherited from AbstractUser)
- `email`: EmailField - Unique, USERNAME_FIELD
- `username`: CharField - Username
- `password`: Hashed password (inherited)
- `account`: ForeignKey to Account - User's account (null=True, blank=True)
- `role`: CharField - User role (choices: developer, owner, admin, editor, viewer, system_bot)
- `created_at`: DateTimeField - Creation timestamp
- `updated_at`: DateTimeField - Last update timestamp
### Role Hierarchy (from highest to lowest)
1. **developer**: Full system access, bypasses all restrictions
2. **owner**: Full access to account, can manage users and billing
3. **admin**: Admin access to account, can manage content and users
4. **editor**: Can edit content, manage clusters/tasks
5. **viewer**: Read-only access
6. **system_bot**: System automation user
### Methods
#### has_role(*roles)
**Parameters**: `*roles` - Variable number of role strings
**Returns**: `True` if user has any of the specified roles
#### is_owner_or_admin()
**Returns**: `True` if role is owner or admin
#### is_developer()
**Returns**: `True` if role is developer or is_superuser
#### is_admin_or_developer()
**Returns**: `True` if role is admin or developer (bypasses restrictions)
**Purpose**: Admin/Developer users bypass account/site/sector filtering
#### is_system_account_user()
**Returns**: `True` if user belongs to system account
**Purpose**: System account users bypass all filtering restrictions
#### get_accessible_sites()
**Returns**: QuerySet of sites user can access
**Access Control Logic**:
- **System Account Users**: All active sites across all accounts
- **Developers**: All active sites across all accounts
- **Owners/Admins**: All sites in their account
- **Editors/Viewers**: Only sites explicitly granted via `SiteUserAccess`
### Relationships
- **Many-to-One**: Account (via `user.account`)
- **Many-to-Many**: Sites (via `SiteUserAccess`)
---
## Plan Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_plans`
### Fields
#### Plan Information
- `id`: Primary key
- `name`: CharField - Plan name (e.g., "Free", "Starter", "Growth")
- `slug`: SlugField - Unique slug identifier
- `price`: DecimalField - Monthly price
- `billing_cycle`: CharField - Billing cycle (choices: monthly, annual)
- `features`: JSONField - Array of feature strings (e.g., ['ai_writer', 'image_gen', 'auto_publish'])
- `is_active`: BooleanField - Whether plan is available for subscription
#### User/Site/Scope Limits
- `max_users`: IntegerField - Total users allowed per account (default: 1, min: 1)
- `max_sites`: IntegerField - Maximum sites allowed (default: 1, min: 1)
- `max_industries`: IntegerField - Optional limit for industries/sectors (nullable)
- `max_author_profiles`: IntegerField - Limit for saved writing styles (default: 5)
#### Planner Limits
- `max_keywords`: IntegerField - Total keywords allowed globally (default: 1000)
- `max_clusters`: IntegerField - Total clusters allowed globally (default: 100)
- `max_content_ideas`: IntegerField - Total content ideas allowed globally (default: 300)
- `daily_cluster_limit`: IntegerField - Max clusters per day (default: 10)
- `daily_keyword_import_limit`: IntegerField - Seed keywords import limit per day (default: 100)
- `monthly_cluster_ai_credits`: IntegerField - AI credits for clustering (default: 50)
#### Writer Limits
- `daily_content_tasks`: IntegerField - Max content tasks per day (default: 10)
- `daily_ai_requests`: IntegerField - Total AI executions per day (default: 50)
- `monthly_word_count_limit`: IntegerField - Monthly word limit for generated content (default: 50000)
- `monthly_content_ai_credits`: IntegerField - AI credit pool for content generation (default: 200)
#### Image Generation Limits
- `monthly_image_count`: IntegerField - Max images per month (default: 100)
- `daily_image_generation_limit`: IntegerField - Max images per day (default: 25)
- `monthly_image_ai_credits`: IntegerField - AI credit pool for images (default: 100)
- `max_images_per_task`: IntegerField - Max images per content task (default: 4, min: 1)
- `image_model_choices`: JSONField - Allowed image models (e.g., ['dalle3', 'hidream'])
#### AI Request Controls
- `daily_ai_request_limit`: IntegerField - Global daily AI request cap (default: 100)
- `monthly_ai_credit_limit`: IntegerField - Unified credit ceiling per month (default: 500)
#### Billing & Credits
- `included_credits`: IntegerField - Monthly credits included (default: 0)
- `extra_credit_price`: DecimalField - Price per additional credit (default: 0.01)
- `allow_credit_topup`: BooleanField - Can user purchase more credits? (default: True)
- `auto_credit_topup_threshold`: IntegerField - Auto top-up trigger point (nullable)
- `auto_credit_topup_amount`: IntegerField - Credits to auto-buy (nullable)
#### Stripe Integration
- `stripe_product_id`: CharField - Stripe product ID (nullable)
- `stripe_price_id`: CharField - Stripe price ID (nullable)
#### Legacy Fields
- `credits_per_month`: IntegerField - DEPRECATED: Use included_credits instead (default: 0)
### Methods
#### clean()
**Purpose**: Validates plan limits
**Validations**:
- `max_sites` must be >= 1
- `included_credits` must be >= 0
#### get_effective_credits_per_month()
**Returns**: `included_credits` if set, otherwise `credits_per_month` (backward compatibility)
### Relationships
- **One-to-Many**: Accounts (via `account.plan`)
### Plan Limits Enforcement
**Current Status**: Plan limits are defined but **not currently enforced** in AI functions.
**Future Implementation**: Plan limits should be checked before allowing operations.
---
## Subscription Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_subscriptions`
### Fields
- `id`: Primary key
- `account`: OneToOneField to Account - Account subscription
- `stripe_subscription_id`: CharField - Unique Stripe subscription ID
- `status`: CharField - Subscription status (choices: active, past_due, canceled, trialing)
- `current_period_start`: DateTimeField - Current billing period start
- `current_period_end`: DateTimeField - Current billing period end
- `cancel_at_period_end`: BooleanField - Cancel at period end flag (default: False)
- `created_at`: DateTimeField - Creation timestamp
- `updated_at`: DateTimeField - Last update timestamp
### Status Values
- `active`: Subscription is active and paid
- `past_due`: Payment failed, subscription past due
- `canceled`: Subscription canceled
- `trialing`: In trial period
### Methods
### Relationships
- **One-to-One**: Account (via `account.subscription`)
---
## Site Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_sites`
**Inherits**: `AccountBaseModel`
### Fields
- `id`: Primary key
- `account`: ForeignKey to Account - Account this site belongs to
- `name`: CharField - Site name
- `slug`: SlugField - Unique slug per account
- `domain`: URLField - Primary domain URL (optional)
- `description`: TextField - Site description (optional)
- `industry`: ForeignKey to Industry - Industry this site belongs to (optional)
- `is_active`: BooleanField - Whether site is active (default: True)
- `status`: CharField - Site status (choices: active, inactive, suspended, default: active)
- `wp_url`: URLField - WordPress site URL (optional, for WordPress integration)
- `wp_username`: CharField - WordPress username (optional)
- `wp_app_password`: CharField - WordPress app password (optional)
- `created_at`: DateTimeField - Creation timestamp
- `updated_at`: DateTimeField - Last update timestamp
### Unique Constraint
- `(account, slug)` - Slug unique per account
### Methods
#### get_active_sectors_count()
**Returns**: Count of active sectors for this site
#### can_add_sector()
**Returns**: `True` if site can add another sector (max 5 sectors per site)
### Relationships
- **Many-to-One**: Account (via `site.account`)
- **Many-to-One**: Industry (via `site.industry`, optional)
- **One-to-Many**: Sectors (via `sector.site`)
- **Many-to-Many**: Users (via `SiteUserAccess`)
### Site Activation
**Important**: Multiple sites can be active simultaneously. The previous restriction of "only 1 site can be active at one time" has been removed.
---
## Sector Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_sectors`
**Inherits**: `AccountBaseModel`
### Fields
- `id`: Primary key
- `account`: ForeignKey to Account - Account this sector belongs to
- `site`: ForeignKey to Site - Site this sector belongs to
- `industry_sector`: ForeignKey to IndustrySector - Reference to industry sector template (optional)
- `name`: CharField - Sector name
- `slug`: SlugField - Unique slug per site
- `description`: TextField - Sector description (optional)
- `is_active`: BooleanField - Whether sector is active (default: True)
- `status`: CharField - Sector status (choices: active, inactive, default: active)
- `created_at`: DateTimeField - Creation timestamp
- `updated_at`: DateTimeField - Last update timestamp
### Unique Constraint
- `(site, slug)` - Slug unique per site
### Validation
**On Save**:
1. Automatically sets `account` from `site.account`
2. Validates that `industry_sector.industry` matches `site.industry` (if both set)
3. Validates sector limit: Maximum 5 active sectors per site
### Methods
#### industry (property)
**Returns**: Industry for this sector (from `industry_sector.industry` if set)
### Relationships
- **Many-to-One**: Account (via `sector.account`)
- **Many-to-One**: Site (via `sector.site`)
- **Many-to-One**: IndustrySector (via `sector.industry_sector`, optional, template reference)
- **One-to-Many**: Keywords, Clusters, ContentIdeas, Tasks (via their `sector` field)
---
## Industry & IndustrySector Models
### Industry Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_industries`
**Purpose**: Global industry templates.
### Fields
- `id`: Primary key
- `name`: CharField - Industry name (unique)
- `slug`: SlugField - Unique slug identifier
- `description`: TextField - Industry description (optional)
- `is_active`: BooleanField - Whether industry is active (default: True)
- `created_at`: DateTimeField - Creation timestamp
- `updated_at`: DateTimeField - Last update timestamp
### Relationships
- **One-to-Many**: Sites (via `site.industry`)
- **One-to-Many**: IndustrySectors (via `industry_sector.industry`)
### IndustrySector Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_industry_sectors`
**Purpose**: Sector templates within industries.
### Fields
- `id`: Primary key
- `industry`: ForeignKey to Industry - Industry this sector belongs to
- `name`: CharField - Sector name
- `slug`: SlugField - Unique slug per industry
- `description`: TextField - Sector description (optional)
- `suggested_keywords`: JSONField - List of suggested keywords for this sector template
- `is_active`: BooleanField - Whether sector is active (default: True)
- `created_at`: DateTimeField - Creation timestamp
- `updated_at`: DateTimeField - Last update timestamp
### Unique Constraint
- `(industry, slug)` - Slug unique per industry
### Relationships
- **Many-to-One**: Industry (via `industry_sector.industry`)
- **One-to-Many**: Sectors (via `sector.industry_sector`, template reference)
- **One-to-Many**: SeedKeywords (via `seed_keyword.sector`)
---
## Access Control
### Account Isolation
**Principle**: All data is isolated by account.
**Implementation**:
- All models inherit `AccountBaseModel` (has `account` ForeignKey)
- All ViewSets inherit `AccountModelViewSet` (filters by `request.account`)
- Middleware sets `request.account` from JWT token
**Access Control**:
- **Admin/Developer users**: Bypass account filtering (see all accounts)
- **System account users**: Bypass account filtering (see all accounts)
- **Regular users**: Only see data from their account
### Site Access Control
**Principle**: Users can only access sites they have permission to access.
**Implementation**:
- `User.get_accessible_sites()` returns sites user can access
- ViewSets filter by accessible sites
- `SiteUserAccess` model controls explicit site access for editors/viewers
**Access Levels**:
- **System Account Users**: All active sites across all accounts
- **Developers**: All active sites across all accounts
- **Owners/Admins**: All sites in their account
- **Editors/Viewers**: Only sites explicitly granted via `SiteUserAccess`
### SiteUserAccess Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_site_user_access`
**Purpose**: Many-to-many relationship between Users and Sites for explicit access control.
### Fields
- `id`: Primary key
- `user`: ForeignKey to User
- `site`: ForeignKey to Site
- `granted_at`: DateTimeField - When access was granted
- `granted_by`: ForeignKey to User - Who granted access (optional)
### Unique Constraint
- `(user, site)` - One access record per user-site pair
### Usage
- Owners and Admins have automatic access to all sites in their account
- Editors and Viewers require explicit `SiteUserAccess` records
- System account users and Developers bypass this check
### Role-Based Access Control (RBAC)
**Role Hierarchy**:
1. **developer**: Full system access, bypasses all restrictions
2. **owner**: Full account access, can manage users and billing
3. **admin**: Account admin access, can manage content and users
4. **editor**: Content editing access, can manage clusters/tasks
5. **viewer**: Read-only access
**Permission Checks**:
- `user.has_role(*roles)`: Check if user has any of the specified roles
- `user.is_owner_or_admin()`: Check if user is owner or admin
- `user.is_developer()`: Check if user is developer
- `user.is_admin_or_developer()`: Check if user is admin or developer (bypasses restrictions)
---
## Credits System
### Account Credits
**Field**: `Account.credits` (IntegerField, default: 0)
**Purpose**: Current credit balance for the account.
**Usage**: Credits are deducted for AI operations and added through purchases or subscriptions.
### Credit Models
#### CreditTransaction
**File**: `backend/igny8_core/modules/billing/models.py`
**Table**: `igny8_credit_transactions`
**Purpose**: Track all credit transactions (additions, deductions).
### Fields
- `id`: Primary key
- `account`: ForeignKey to Account
- `transaction_type`: CharField - Type of transaction (choices: purchase, subscription, refund, deduction, adjustment)
- `amount`: IntegerField - Positive for additions, negative for deductions
- `balance_after`: IntegerField - Credit balance after this transaction
- `description`: CharField - Transaction description
- `metadata`: JSONField - Additional context (AI call details, etc.)
- `created_at`: DateTimeField - Creation timestamp
#### CreditUsageLog
**File**: `backend/igny8_core/modules/billing/models.py`
**Table**: `igny8_credit_usage_logs`
**Purpose**: Detailed log of credit usage per AI operation.
### Fields
- `id`: Primary key
- `account`: ForeignKey to Account
- `operation_type`: CharField - Type of operation (choices: clustering, ideas, content, images, reparse)
- `credits_used`: IntegerField - Number of credits used
- `cost_usd`: DecimalField - Cost in USD (optional)
- `model_used`: CharField - AI model used (optional)
- `tokens_input`: IntegerField - Input tokens (optional)
- `tokens_output`: IntegerField - Output tokens (optional)
- `related_object_type`: CharField - Related object type (optional)
- `related_object_id`: IntegerField - Related object ID (optional)
- `metadata`: JSONField - Additional metadata
- `created_at`: DateTimeField - Creation timestamp
### CreditService
**File**: `backend/igny8_core/modules/billing/services.py`
**Purpose**: Service for managing credits.
#### Methods
##### check_credits(account, required_credits)
**Purpose**: Check if account has enough credits.
**Raises**: `InsufficientCreditsError` if account doesn't have enough credits
##### deduct_credits(account, amount, operation_type, description, ...)
**Purpose**: Deduct credits and log transaction.
**Parameters**:
- `account`: Account instance
- `amount`: Number of credits to deduct
- `operation_type`: Type of operation
- `description`: Description of the transaction
- `metadata`: Optional metadata dict
- `cost_usd`: Optional cost in USD
- `model_used`: Optional AI model used
- `tokens_input`: Optional input tokens
- `tokens_output`: Optional output tokens
- `related_object_type`: Optional related object type
- `related_object_id`: Optional related object ID
**Returns**: New credit balance
**Creates**:
- `CreditTransaction` record
- `CreditUsageLog` record
##### add_credits(account, amount, transaction_type, description, ...)
**Purpose**: Add credits (purchase, subscription, etc.).
**Parameters**:
- `account`: Account instance
- `amount`: Number of credits to add
- `transaction_type`: Type of transaction
- `description`: Description of the transaction
- `metadata`: Optional metadata dict
**Returns**: New credit balance
**Creates**: `CreditTransaction` record
##### calculate_credits_for_operation(operation_type, **kwargs)
**Purpose**: Calculate credits needed for an operation.
**Parameters**:
- `operation_type`: Type of operation
- `**kwargs`: Operation-specific parameters
**Returns**: Number of credits required
**Raises**: `CreditCalculationError` if calculation fails
**Credit Costs**:
- `clustering`: 1 credit per 30 keywords
- `ideas`: 1 credit per idea
- `content`: 3 credits per content piece
- `images`: 1 credit per image
- `reparse`: 1 credit per reparse
### Credit Integration
**Current Status**: Credits system is implemented but **not currently enforced** in AI functions.
**Future Implementation**: Credits should be checked and deducted before allowing AI operations.
---
## Plan Limits
### Planner Limits
- `max_keywords`: Total keywords allowed (not enforced)
- `max_clusters`: Total clusters allowed (not enforced)
- `max_content_ideas`: Total content ideas allowed (not enforced)
- `daily_cluster_limit`: Max clusters per day (not enforced)
- `monthly_cluster_ai_credits`: AI credits for clustering (not enforced)
### Writer Limits
- `daily_content_tasks`: Max content tasks per day (not enforced)
- `daily_ai_requests`: Total AI executions per day (not enforced)
- `monthly_word_count_limit`: Monthly word limit (not enforced)
- `monthly_content_ai_credits`: AI credits for content (not enforced)
### Image Limits
- `monthly_image_count`: Max images per month (not enforced)
- `daily_image_generation_limit`: Max images per day (not enforced)
- `monthly_image_ai_credits`: AI credits for images (not enforced)
- `max_images_per_task`: Max images per task (not enforced)
- `image_model_choices`: Allowed image models (not enforced)
### Feature Flags
- `features`: JSON array of enabled features (e.g., `['ai_writer', 'image_gen', 'auto_publish']`)
- **Not enforced**: Feature flags are not checked before allowing AI operations
### Plan Limits Enforcement
**Current Status**: Plan limits are defined but **not currently enforced** in AI functions.
**Future Implementation**: Plan limits should be checked before allowing operations.
---
## Multi-Tenancy Architecture
### Account Isolation
**Principle**: All data is isolated by account.
**Implementation**:
1. **Model Level**: All models inherit `AccountBaseModel` (has `account` ForeignKey)
2. **ViewSet Level**: All ViewSets inherit `AccountModelViewSet` (filters by `request.account`)
3. **Middleware Level**: `AccountContextMiddleware` sets `request.account` from JWT token
**Access Control**:
- Admin/Developer users: Bypass account filtering (see all accounts)
- System account users: Bypass account filtering (see all accounts)
- Regular users: Only see data from their account
### Site/Sector Hierarchy
**Structure**:
```
Account (1) ──< (N) Site
Site (1) ──< (1-5) Sector
Sector (1) ──< (N) Keywords, Clusters, ContentIdeas, Tasks
```
**Implementation**:
- Models inherit `SiteSectorBaseModel` (has `site` and `sector` ForeignKeys)
- ViewSets inherit `SiteSectorModelViewSet` (filters by accessible sites)
- User access control via `User.get_accessible_sites()`
**Site Access Control**:
- System account users: All active sites
- Developers: All active sites
- Owners/Admins: All sites in their account
- Editors/Viewers: Only sites granted via `SiteUserAccess`
### Data Isolation Flow
1. **Request Arrives**: JWT token contains account ID
2. **Middleware**: `AccountContextMiddleware` extracts account ID and sets `request.account`
3. **ViewSet**: `AccountModelViewSet.get_queryset()` filters by `request.account`
4. **Database**: All queries automatically filtered by account
**Override**: Admin/Developer users bypass account filtering
---
## Summary
The IGNY8 multi-tenancy architecture provides:
1. **Complete Account Isolation**: All data isolated by account with automatic filtering
2. **Site/Sector Hierarchy**: Hierarchical organization with access control
3. **Role-Based Access Control**: Granular permissions based on user roles
4. **Plan-Based Limits**: Comprehensive plan limits (defined but not enforced)
5. **Credits System**: Credit tracking and deduction system (implemented but not enforced)
6. **Flexible Site Access**: Multiple sites can be active simultaneously
7. **Industry Templates**: Global industry and sector templates
This architecture ensures data security, scalability, and flexibility while providing a solid foundation for subscription management and billing.

1420
docs/ActiveDocs/06-INFRASTRUCTURE-DEPLOYMENT.md Normal file
View File

File diff suppressed because it is too large Load Diff

274
docs/ActiveDocs/07-AI-FRAMEWORK.md Normal file
View File

@@ -0,0 +1,274 @@
# IGNY8 AI Framework Documentation
**Version:** 1.0
**Last Updated:** 2025-01-XX
**Purpose:** Complete documentation of the unified AI framework architecture.
---
## Overview
The IGNY8 AI Framework provides a unified, consistent architecture for all AI functions. It eliminates code duplication, standardizes progress tracking, and provides a single interface for all AI operations.
### Key Benefits
- **90% Code Reduction**: Functions are now ~100 lines instead of ~600
- **Consistent UX**: All functions use the same progress modal and tracking
- **Unified Logging**: Single `AITaskLog` table for all AI operations
- **Easy Extension**: Add new functions by creating one class
- **Better Debugging**: Detailed step-by-step tracking for all operations
---
## Architecture
### Directory Structure
```
igny8_core/ai/
├── __init__.py # Auto-registers all functions
├── apps.py # Django app configuration
├── admin.py # Admin interface for AITaskLog
├── base.py # BaseAIFunction abstract class
├── engine.py # AIEngine orchestrator
├── processor.py # AIProcessor wrapper
├── registry.py # Function registry
├── tracker.py # StepTracker, ProgressTracker, CostTracker
├── tasks.py # Unified Celery task entrypoint
├── types.py # Shared dataclasses
├── models.py # AITaskLog model
└── functions/ # Function implementations
├── __init__.py
└── auto_cluster.py # Auto cluster function
```
---
## Core Components
### 1. BaseAIFunction
Abstract base class that all AI functions inherit from.
**Methods to implement:**
- `get_name()`: Return function name
- `prepare()`: Load and prepare data
- `build_prompt()`: Build AI prompt
- `parse_response()`: Parse AI response
- `save_output()`: Save results to database
**Optional overrides:**
- `validate()`: Custom validation
- `get_max_items()`: Set item limit
- `get_model()`: Specify AI model
- `get_metadata()`: Function metadata
### 2. AIEngine
Central orchestrator that manages the execution pipeline.
**Phases:**
- INIT (0-10%): Validation & setup
- PREP (10-25%): Data loading & prompt building
- AI_CALL (25-60%): API call to provider
- PARSE (60-80%): Response parsing
- SAVE (80-95%): Database operations
- DONE (95-100%): Finalization
### 3. Function Registry
Dynamic function discovery system.
**Usage:**
```python
from igny8_core.ai.registry import register_function, get_function
# Register function
register_function('auto_cluster', AutoClusterFunction)
# Get function
fn = get_function('auto_cluster')
```
### 4. Unified Celery Task
Single entrypoint for all AI functions.
**Endpoint:** `run_ai_task(function_name, payload, account_id)`
**Example:**
```python
from igny8_core.ai.tasks import run_ai_task
task = run_ai_task.delay(
function_name='auto_cluster',
payload={'ids': [1, 2, 3], 'sector_id': 1},
account_id=1
)
```
---
## Function Implementation Example
### Auto Cluster Function
```python
from igny8_core.ai.base import BaseAIFunction
class AutoClusterFunction(BaseAIFunction):
def get_name(self) -> str:
return 'auto_cluster'
def get_max_items(self) -> int:
return 20
def prepare(self, payload: dict, account=None) -> Dict:
# Load keywords
ids = payload.get('ids', [])
keywords = Keywords.objects.filter(id__in=ids)
return {'keywords': keywords, ...}
def build_prompt(self, data: Dict, account=None) -> str:
# Build clustering prompt
return prompt_template.replace('[IGNY8_KEYWORDS]', keywords_text)
def parse_response(self, response: str, step_tracker=None) -> List[Dict]:
# Parse AI response
return clusters
def save_output(self, parsed, original_data, account, progress_tracker) -> Dict:
# Save clusters to database
return {'clusters_created': 5, 'keywords_updated': 20}
```
---
## API Endpoint Example
### Before (Old): ~300 lines
### After (New): ~50 lines
```python
@action(detail=False, methods=['post'], url_path='auto_cluster')
def auto_cluster(self, request):
from igny8_core.ai.tasks import run_ai_task
account = getattr(request, 'account', None)
account_id = account.id if account else None
payload = {
'ids': request.data.get('ids', []),
'sector_id': request.data.get('sector_id')
}
task = run_ai_task.delay(
function_name='auto_cluster',
payload=payload,
account_id=account_id
)
return Response({
'success': True,
'task_id': str(task.id),
'message': 'Clustering started'
})
```
---
## Progress Tracking
### Unified Progress Endpoint
**URL:** `/api/v1/system/settings/task_progress/<task_id>/`
**Response:**
```json
{
"state": "PROGRESS",
"meta": {
"phase": "AI_CALL",
"percentage": 45,
"message": "Analyzing keyword relationships...",
"request_steps": [...],
"response_steps": [...],
"cost": 0.000123,
"tokens": 1500
}
}
```
### Frontend Integration
All AI functions use the same progress modal:
- Single `useProgressModal` hook
- Unified progress endpoint
- Consistent phase labels
- Step-by-step logs
---
## Database Logging
### AITaskLog Model
Unified logging table for all AI operations.
**Fields:**
- `task_id`: Celery task ID
- `function_name`: Function name
- `account`: Account (required)
- `phase`: Current phase
- `status`: success/error/pending
- `cost`: API cost
- `tokens`: Token usage
- `request_steps`: Request step logs
- `response_steps`: Response step logs
- `error`: Error message (if any)
---
## Migration Guide
### Migrating Existing Functions
1. Create function class inheriting `BaseAIFunction`
2. Implement required methods
3. Register function in `ai/__init__.py`
4. Update API endpoint to use `run_ai_task`
5. Test and remove old code
### Example Migration
**Old code:**
```python
@action(...)
def auto_cluster(self, request):
# 300 lines of code
```
**New code:**
```python
@action(...)
def auto_cluster(self, request):
# 20 lines using framework
```
---
## Summary
The AI Framework provides:
1. **Unified Architecture**: Single framework for all AI functions
2. **Code Reduction**: 90% less code per function
3. **Consistent UX**: Same progress modal for all functions
4. **Better Debugging**: Detailed step tracking
5. **Easy Extension**: Add functions quickly
6. **Unified Logging**: Single log table
7. **Cost Tracking**: Automatic cost calculation
This architecture ensures maintainability, consistency, and extensibility while dramatically reducing code duplication.