salman/igny8
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

DOCS

Browse Source
This commit is contained in:
Desktop
2025-11-12 20:22:08 +05:00
parent 14534dc3ee
commit e4a6bd1160
16 changed files with 2519 additions and 6392 deletions
Download Patch File Download Diff File Expand all files Collapse all files

45
README.md
View File

@@ -81,39 +81,50 @@ docker-compose -f docker-compose.app.yml up
### ✅ Implemented ### ✅ Implemented
- **Foundation**: Multi-tenancy system, Authentication (login/register), RBAC permissions - **Foundation**: Multi-tenancy system, Authentication (login/register), RBAC permissions
- **Planner Module**: Keywords & Clusters (full CRUD, filtering, pagination, bulk operations, CSV import/export) - **Planner Module**: Keywords, Clusters, Content Ideas (full CRUD, filtering, pagination, bulk operations, CSV import/export)
- **Writer Module**: Tasks, Content, Images (full CRUD, content generation, image generation)
- **Thinker Module**: Prompts, Author Profiles, Strategies, Image Testing
- **System Module**: Settings, Integrations (OpenAI, Runware), AI Prompts
- **Billing Module**: Credits, Transactions, Usage Logs
- **AI Functions**: 5 AI operations (Auto Cluster, Generate Ideas, Generate Content, Generate Image Prompts, Generate Images)
- **Frontend**: Complete component library, 4 master templates, config-driven UI system - **Frontend**: Complete component library, 4 master templates, config-driven UI system
- **Backend**: REST API with tenant isolation, Site > Sector hierarchy - **Backend**: REST API with tenant isolation, Site > Sector hierarchy, Celery async tasks
- **Development**: Docker Compose setup, hot reload, TypeScript + React - **Development**: Docker Compose setup, hot reload, TypeScript + React
### 🚧 In Progress ### 🚧 In Progress
- Content Ideas module (backend + frontend)
- AI integration for auto-clustering and idea generation
- Planner Dashboard enhancement with KPIs - Planner Dashboard enhancement with KPIs
- WordPress integration (publishing)
- Automation & CRON tasks
### 🔄 Planned ### 🔄 Planned
- Writer module (Tasks, Drafts, Published) - Analytics module enhancements
- Thinker module (Prompts, Strategies, Image Testing) - Advanced scheduling features
- AI Pipeline infrastructure - Additional AI model integrations
- WordPress integration
- Automation & CRON tasks
## 🔗 API Endpoints ## 🔗 API Endpoints
- **Keywords**: `/api/planner/keywords/` - **Planner**: `/api/v1/planner/keywords/`, `/api/v1/planner/clusters/`, `/api/v1/planner/ideas/`
- **Writer**: `/api/v1/writer/tasks/`, `/api/v1/writer/images/`
- **System**: `/api/v1/system/settings/`
- **Billing**: `/api/v1/billing/`
- **Admin**: `/admin/` - **Admin**: `/admin/`
See `docs/04-BACKEND.md` for complete API reference.
## 📖 Documentation ## 📖 Documentation
All documentation is consolidated in the `/docs/` folder: All documentation is consolidated in the `/docs/` folder. Start with `docs/README.md` for the complete documentation index.
- **`docs/01-ARCHITECTURE.md`** - System architecture, design patterns, and key principles ### Core Documentation
- **`docs/02-IMPLEMENTATION-ROADMAP.md`** - Complete build roadmap with 21 phases - **`docs/README.md`** - Documentation index and navigation
- **`docs/03-CURRENT-STATUS.md`** - Current progress, completed items, and next steps - **`docs/01-ARCHITECTURE-TECH-STACK.md`** - Technology stack and system architecture
- **`docs/04-API-REFERENCE.md`** - API endpoints reference guide - **`docs/02-APP-ARCHITECTURE.md`** - Application architecture with complete workflows
- **`docs/05-WP-MIGRATION-MAP.md`** - WordPress plugin to Django app migration reference - **`docs/03-FRONTEND.md`** - Complete frontend documentation
- **`docs/04-BACKEND.md`** - Complete backend documentation
- **`docs/05-AI-FUNCTIONS.md`** - Complete AI functions documentation
- **`docs/06-CHANGELOG.md`** - System changelog
**Quick Start**: Read `docs/03-CURRENT-STATUS.md` for current state, then `docs/02-IMPLEMENTATION-ROADMAP.md` for what to build next. **Quick Start**: Read `docs/README.md` for navigation, then start with `docs/01-ARCHITECTURE-TECH-STACK.md` for system overview.
## 🛠️ Development ## 🛠️ Development

476
docs/01-ARCHITECTURE-TECH-STACK.md Normal file
View File

@@ -0,0 +1,476 @@
# IGNY8 Architecture & Technology Stack
**Last Updated:** 2025-01-XX
**Purpose:** Complete technology stack and architecture overview for the IGNY8 platform.
---
## Table of Contents
1. [Executive Summary](#executive-summary)
2. [Technology Stack](#technology-stack)
3. [System Architecture](#system-architecture)
4. [Core Architecture Principles](#core-architecture-principles)
5. [Infrastructure Components](#infrastructure-components)
6. [External Service Integrations](#external-service-integrations)
7. [Deployment Architecture](#deployment-architecture)
---
## Executive Summary
**IGNY8** is a full-stack SaaS platform for SEO keyword management and AI-driven content generation. The system is built with modern technologies and follows a multi-tenant architecture with complete account isolation.
### Key Metrics
- **Architecture**: Multi-tenant SaaS with account isolation
- **Backend**: Django 5.2+ with Django REST Framework
- **Frontend**: React 19 with TypeScript
- **Database**: PostgreSQL 15
- **Task Queue**: Celery with Redis
- **Deployment**: Docker-based containerization
- **Reverse Proxy**: Caddy (HTTPS termination)
- **AI Functions**: 5 primary AI operations
- **Modules**: 4 core modules (Planner, Writer, System, Billing)
---
## Technology Stack
### Backend Stack
| Component | Technology | Version | Purpose |
|-----------|------------|---------|---------|
| **Framework** | Django | 5.2+ | Web framework |
| **API Framework** | Django REST Framework | Latest | RESTful API |
| **Database** | PostgreSQL | 15 | Primary database |
| **Task Queue** | Celery | Latest | Asynchronous tasks |
| **Cache/Broker** | Redis | 7 | Celery broker & caching |
| **Authentication** | JWT | SimpleJWT | Token-based auth |
| **HTTP Client** | Requests | Latest | External API calls |
| **WSGI Server** | Gunicorn | Latest | Production server |
### Frontend Stack
| Component | Technology | Version | Purpose |
|-----------|------------|---------|---------|
| **Framework** | React | 19 | UI library |
| **Language** | TypeScript | Latest | Type safety |
| **Build Tool** | Vite | Latest | Build tool & dev server |
| **Styling** | Tailwind CSS | Latest | Utility-first CSS |
| **State Management** | Zustand | Latest | Lightweight state |
| **Routing** | React Router | v6 | Client-side routing |
| **HTTP Client** | Fetch API | Native | API communication |
### Infrastructure Stack
| Component | Technology | Purpose |
|-----------|------------|---------|
| **Containerization** | Docker | Application containers |
| **Orchestration** | Docker Compose | Multi-container orchestration |
| **Reverse Proxy** | Caddy | HTTPS termination & routing |
| **Database Admin** | pgAdmin | PostgreSQL administration |
| **File Management** | FileBrowser | Web-based file management |
| **Container Management** | Portainer | Docker container management |
### External Services
| Service | Purpose | Integration |
|---------|---------|-------------|
| **OpenAI API** | Text generation (GPT models) | API integration |
| **OpenAI DALL-E** | Image generation | API integration |
| **Runware API** | Alternative image generation | API integration |
| **WordPress** | Content publishing | REST API integration |
| **Stripe** | Payment processing | Webhook integration (planned) |
---
## System Architecture
### High-Level Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Client Layer │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Browser │ │ Mobile │ │ Admin │ │
│ │ (React) │ │ (Future) │ │ Panel │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
└─────────┼──────────────────┼──────────────────┼─────────────┘
│ │ │
└──────────────────┼──────────────────┘
│
┌────────────────────────────┼──────────────────────────────┐
│ Reverse Proxy Layer │
│ ┌───────────────┐ │
│ │ Caddy │ │
│ │ (HTTPS/443) │ │
│ └───────┬───────┘ │
└────────────────────────────┼──────────────────────────────┘
│
┌────────────────────────────┼──────────────────────────────┐
│ Application Layer │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ Frontend │ │ Backend │ │
│ │ (React) │◄─────────────┤ (Django) │ │
│ │ Port 8021 │ REST API │ Port 8011 │ │
│ └──────────────┘ └──────┬───────┘ │
│ │ │
│ ┌────────┴────────┐ │
│ │ Celery Worker │ │
│ │ (Async Tasks) │ │
│ └────────┬────────┘ │
└───────────────────────────────────────┼──────────────────┘
│
┌───────────────────────────────────────┼──────────────────┐
│ Data Layer │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ PostgreSQL │ │ Redis │ │ Storage │ │
│ │ (Database) │ │ (Cache/Broker)│ │ (Files) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└──────────────────────────────────────────────────────────┘
│
┌───────────────────────────────────────┼──────────────────┐
│ External Services │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ OpenAI │ │ Runware │ │ WordPress │ │
│ │ (GPT/DALL-E)│ │ (Images) │ │ (Publish) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
└──────────────────────────────────────────────────────────┘
```
### Request Flow
```
1. User Request
↓
2. Browser (React Frontend)
↓
3. Caddy Reverse Proxy (HTTPS Termination)
↓
4. Django Backend (API Endpoint)
↓
5. AccountContextMiddleware (Account Isolation)
↓
6. ViewSet (Business Logic)
↓
7. Serializer (Validation)
↓
8. Model (Database)
↓
9. Response (JSON)
↓
10. Frontend (UI Update)
```
### AI Task Flow
```
1. User Action (e.g., "Auto Cluster Keywords")
↓
2. Frontend API Call
↓
3. Backend Endpoint (ViewSet Action)
↓
4. Celery Task Queued
↓
5. Task ID Returned to Frontend
↓
6. Frontend Polls Progress Endpoint
↓
7. Celery Worker Processes Task
↓
8. AIProcessor Makes API Calls
↓
9. Results Saved to Database
↓
10. Progress Updates Sent
↓
11. Frontend Displays Results
```
---
## Core Architecture Principles
### 1. Configuration-Driven Everything
**Principle**: Zero HTML/JSX duplication - All UI rendered from configuration.
**Implementation**:
- **Frontend**: Config files in `/config/pages/` and `/config/snippets/`
- **Backend**: DRF serializers and ViewSet actions
- **Templates**: 4 universal templates (Dashboard, Table, Form, System)
**Benefits**:
- Single source of truth
- Easy maintenance
- Consistent UI/UX
- Rapid feature development
### 2. Multi-Tenancy Foundation
**Principle**: Complete account isolation with automatic filtering.
**Implementation**:
- All models inherit `AccountBaseModel`
- All ViewSets inherit `AccountModelViewSet`
- Middleware injects account context from JWT
- Site > Sector hierarchy for content organization
**Benefits**:
- Data security
- Scalability
- Resource isolation
- Simplified access control
### 3. Template System (4 Universal Templates)
**Principle**: Reusable templates for all page types.
| Template | Purpose | Usage |
|----------|---------|-------|
| **DashboardTemplate** | Module home pages | KPIs, workflow steps, charts |
| **TablePageTemplate** | CRUD table pages | Keywords, Clusters, Tasks, etc. |
| **FormPageTemplate** | Settings/form pages | Settings, Integration, etc. |
| **SystemPageTemplate** | System/admin pages | Logs, Status, Monitoring |
### 4. Unified AI Processor
**Principle**: Single interface for all AI operations.
**Implementation**:
- Single `AIEngine` class orchestrates all AI operations
- All AI functions inherit from `BaseAIFunction`
- Manual and automated workflows use same functions
- Account-specific API keys and model configuration
**Benefits**:
- Code reusability
- Consistent error handling
- Unified logging
- Easy to extend
### 5. Module-Based Organization
**Principle**: Clear module boundaries with shared utilities.
**Modules**:
- **Planner**: Keywords, Clusters, Ideas
- **Writer**: Tasks, Content, Images
- **System**: Settings, Prompts, Integration
- **Billing**: Credits, Transactions, Usage
- **Auth**: Accounts, Users, Sites, Sectors
---
## Infrastructure Components
### Docker Architecture
The system uses a two-stack Docker architecture:
1. **Infrastructure Stack (`igny8-infra`)**: Shared services
2. **Application Stack (`igny8-app`)**: Application-specific services
### Infrastructure Stack Services
| Service | Container Name | Port | Purpose |
|---------|----------------|------|---------|
| **PostgreSQL** | `igny8_postgres` | 5432 (internal) | Database |
| **Redis** | `igny8_redis` | 6379 (internal) | Cache & Celery broker |
| **pgAdmin** | `igny8_pgadmin` | 5050:80 | Database administration |
| **FileBrowser** | `igny8_filebrowser` | 8080:80 | File management |
| **Caddy** | `igny8_caddy` | 80:80, 443:443 | Reverse proxy & HTTPS |
| **Setup Helper** | `setup-helper` | - | Utility container |
### Application Stack Services
| Service | Container Name | Port | Purpose |
|---------|----------------|------|---------|
| **Backend** | `igny8_backend` | 8011:8010 | Django REST API |
| **Frontend** | `igny8_frontend` | 8021:5173 | React application |
| **Celery Worker** | `igny8_celery_worker` | - | Async task processing |
| **Celery Beat** | `igny8_celery_beat` | - | Scheduled tasks |
### Network Configuration
- **Network Name**: `igny8_net`
- **Type**: External bridge network
- **Purpose**: Inter-container communication
- **Creation**: Must be created manually before starting stacks
### Volume Management
**Infrastructure Volumes**:
- `pgdata`: PostgreSQL database data
- `redisdata`: Redis data
- `pgadmin_data`: pgAdmin configuration
- `filebrowser_db`: FileBrowser database
- `caddy_data`: Caddy SSL certificates
- `caddy_config`: Caddy configuration
**Application Volumes**:
- Host mounts for application code
- Host mounts for logs
- Docker socket for container management
### Port Allocation
| Service | External Port | Internal Port | Access |
|---------|---------------|---------------|--------|
| **pgAdmin** | 5050 | 80 | http://localhost:5050 |
| **FileBrowser** | 8080 | 80 | http://localhost:8080 |
| **Caddy** | 80, 443 | 80, 443 | https://domain.com |
| **Backend** | 8011 | 8010 | http://localhost:8011 |
| **Frontend** | 8021 | 5173 | http://localhost:8021 |
---
## External Service Integrations
### OpenAI Integration
**Purpose**: Text generation and image generation
**Services Used**:
- GPT models for text generation
- DALL-E for image generation
**Configuration**:
- API key stored per account in `IntegrationSettings`
- Model selection per account
- Cost tracking per request
### Runware Integration
**Purpose**: Alternative image generation service
**Configuration**:
- API key stored per account
- Model selection (e.g., `runware:97@1`)
- Image type selection (realistic, artistic, cartoon)
### WordPress Integration
**Purpose**: Content publishing
**Configuration**:
- WordPress URL per site
- Username and password stored per site
- REST API integration for publishing
**Workflow**:
1. Content generated in IGNY8
2. Images attached
3. Content published to WordPress via REST API
4. Status updated in IGNY8
### Stripe Integration (Planned)
**Purpose**: Payment processing
**Status**: Planned for future implementation
**Features**:
- Subscription management
- Payment processing
- Webhook integration
---
## Deployment Architecture
### Deployment Model
**Container-Based**: All services run in Docker containers
**Stack Separation**:
- Infrastructure stack runs independently
- Application stack depends on infrastructure stack
- Both stacks share the same Docker network
### Environment Configuration
**Backend Environment Variables**:
- Database connection (PostgreSQL)
- Redis connection
- Django settings (DEBUG, SECRET_KEY, etc.)
- JWT settings
- Celery configuration
**Frontend Environment Variables**:
- Backend API URL
- Environment (development/production)
**Infrastructure Environment Variables**:
- PostgreSQL credentials
- pgAdmin credentials
- FileBrowser configuration
### Health Checks
**Backend Health Check**:
- Endpoint: `/api/v1/system/status/`
- Interval: 30 seconds
- Timeout: 10 seconds
- Retries: 3
**PostgreSQL Health Check**:
- Command: `pg_isready`
- Interval: 20 seconds
- Timeout: 3 seconds
- Retries: 5
**Redis Health Check**:
- Command: `redis-cli ping`
- Interval: 20 seconds
- Timeout: 3 seconds
- Retries: 5
### Scaling Considerations
**Horizontal Scaling**:
- Multiple Celery workers can be added
- Multiple backend instances can be added (with load balancer)
- Frontend can be scaled independently
**Vertical Scaling**:
- Database can be scaled with more resources
- Redis can be scaled with more memory
- Containers can be allocated more CPU/memory
### Backup & Recovery
**Database Backups**:
- PostgreSQL dumps stored in `/data/backups`
- Automated backup scripts
- Point-in-time recovery support
**Volume Backups**:
- Docker volume backups
- Application code backups
- Configuration backups
---
## Summary
The IGNY8 platform is built on a modern, scalable architecture using:
- **Django 5.2+** for the backend API
- **React 19** for the frontend
- **PostgreSQL 15** for data storage
- **Celery & Redis** for async processing
- **Docker** for containerization
- **Caddy** for reverse proxy and HTTPS
The architecture follows principles of:
- Configuration-driven development
- Multi-tenancy with account isolation
- Module-based organization
- Unified AI processing
- Template-based UI rendering
This architecture supports scalability, maintainability, and rapid feature development while ensuring data security and isolation.

795
docs/02-APP-ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,795 @@
# IGNY8 Application Architecture
**Last Updated:** 2025-01-XX
**Purpose:** Complete application architecture documentation covering system hierarchy, modules, workflows, features, and data flow.
---
## Table of Contents
1. [System Overview](#system-overview)
2. [System Hierarchy](#system-hierarchy)
3. [User Roles & Access Control](#user-roles--access-control)
4. [Module Organization](#module-organization)
5. [Complete Workflows](#complete-workflows)
6. [Data Models & Relationships](#data-models--relationships)
7. [Multi-Tenancy Architecture](#multi-tenancy-architecture)
8. [API Architecture](#api-architecture)
9. [Security Architecture](#security-architecture)
10. [Integration Points](#integration-points)
---
## System Overview
**IGNY8** is a full-stack SaaS platform for SEO keyword management and AI-driven content generation. The system provides a scalable, multi-account platform that enables users to manage keywords, generate content ideas, create AI-powered content, and publish to WordPress.
### Platform Capabilities
| Capability | Description | Module |
|------------|-------------|--------|
| **Keyword Management** | Import, organize, filter, and manage SEO keywords | Planner |
| **Keyword Clustering** | AI-powered semantic clustering of related keywords | Planner |
| **Content Ideas** | Generate content ideas from keyword clusters | Planner |
| **Content Generation** | AI-powered blog post and article generation | Writer |
| **Image Generation** | AI-powered image generation (DALL-E, Runware) | Writer |
| **WordPress Integration** | Direct publishing to WordPress sites | Writer |
| **Account Management** | Multi-account SaaS with user roles | Auth |
| **Billing & Credits** | Credit-based billing system | Billing |
| **AI Configuration** | Customizable AI prompts and settings | System |
---
## System Hierarchy
### Entity Relationships
```
Account (1) ──< (N) User
Account (1) ──< (1) Subscription ──> (1) Plan
Account (1) ──< (N) Site
Site (1) ──< (1-5) Sector
Sector (1) ──< (N) Keywords, Clusters, ContentIdeas, Tasks
Cluster (1) ──< (N) Keywords (Many-to-Many)
Cluster (1) ──< (N) ContentIdeas
ContentIdeas (1) ──< (N) Tasks
Task (1) ──> (1) Content
Task (1) ──< (N) Images
```
### Hierarchy Details
**Account Level**:
- Top-level organization/workspace
- Contains users, sites, subscriptions, and all data
- Has credit balance and plan assignment
- Status: active, suspended, trial, cancelled
**User Level**:
- Individual user accounts within an account
- Has role (developer, owner, admin, editor, viewer)
- Can belong to only one account
- Access controlled by role and site permissions
**Site Level**:
- Workspace within an account (1-N relationship)
- Can have multiple active sites simultaneously
- Has WordPress integration settings (URL, username, password)
- Can be associated with an industry
- Status: active, inactive, suspended
**Sector Level**:
- Content category within a site (1-5 per site)
- Organizes keywords, clusters, ideas, and tasks
- Can reference an industry sector template
- Status: active, inactive
**Content Level**:
- Keywords, Clusters, ContentIdeas belong to Sector
- Tasks, Content, Images belong to Sector
- All content is automatically associated with Account and Site
---
## User Roles & Access Control
### User Roles
| Role | Access Level | Description | Permissions |
|------|--------------|-------------|-------------|
| **Developer** | System-wide | Full system access, bypasses all restrictions | All accounts, all sites, all data, system settings |
| **Owner** | Account-wide | Full account access, can manage users and billing | All sites in account, user management, billing |
| **Admin** | Account-wide | Account admin access, can manage content and users | All sites in account, content management, user management |
| **Editor** | Site-specific | Content editing access, can manage clusters/tasks | Granted sites only, content editing |
| **Viewer** | Site-specific | Read-only access | Granted sites only, read-only |
| **System Bot** | System-wide | System automation user | All accounts, all sites, all data |
### Access Control Matrix
| User Role | Account Access | Site Access | Data Access | User Management | Billing |
|-----------|----------------|-------------|-------------|------------------|---------|
| Developer | All accounts | All sites | All data | Yes | Yes |
| System Bot | All accounts | All sites | All data | No | No |
| Owner | Own account | All sites in account | All data in account | Yes | Yes |
| Admin | Own account | All sites in account | All data in account | Yes | No |
| Editor | Own account | Granted sites only | Data in granted sites | No | No |
| Viewer | Own account | Granted sites only | Read-only in granted sites | No | No |
### Site Access Control
**Automatic Access**:
- Owners and Admins: Automatic access to all sites in their account
- Developers and System Bot: Access to all sites across all accounts
**Explicit Access**:
- Editors and Viewers: Require explicit `SiteUserAccess` records
- Access granted by Owner or Admin
- Access can be revoked at any time
### Account Isolation
**Principle**: All data is automatically filtered by account.
**Implementation**:
- All models inherit `AccountBaseModel` with `account` ForeignKey
- All ViewSets inherit `AccountModelViewSet` with automatic filtering
- Middleware sets `request.account` from JWT token
- Queries automatically filter by account (except for Developer/System Bot)
---
## Module Organization
### Module Structure
| Module | Purpose | Models | ViewSets | Celery Tasks |
|--------|---------|--------|----------|--------------|
| **Planner** | Keyword management & content planning | Keywords, Clusters, ContentIdeas | KeywordViewSet, ClusterViewSet, ContentIdeasViewSet | auto_cluster_keywords_task, auto_generate_ideas_task |
| **Writer** | Content generation & management | Tasks, Content, Images | TasksViewSet, ImagesViewSet | auto_generate_content_task, auto_generate_images_task |
| **System** | Settings, prompts, integrations | AIPrompt, IntegrationSettings, AuthorProfile, Strategy | AIPromptViewSet, IntegrationSettingsViewSet, AuthorProfileViewSet | - |
| **Billing** | Credits, transactions, usage | CreditTransaction, CreditUsageLog | CreditTransactionViewSet, CreditUsageLogViewSet | - |
| **Auth** | Multi-tenancy, users, accounts | Account, User, Plan, Site, Sector | AccountViewSet, UserViewSet, SiteViewSet, SectorViewSet | - |
### Module Dependencies
```
Auth (Core)
├── Planner (depends on Auth)
├── Writer (depends on Auth, Planner)
├── System (depends on Auth)
└── Billing (depends on Auth)
```
### Module Features
#### Planner Module
**Purpose**: Keyword research, clustering, and content idea generation
**Features**:
- Keyword import (CSV/manual)
- Keyword filtering and organization
- AI-powered keyword clustering
- Content idea generation from clusters
- Keyword-to-cluster mapping
- Cluster metrics and analytics
**Models**:
- `Keywords`: Individual keywords with metrics (volume, difficulty, intent)
- `Clusters`: Groups of related keywords
- `ContentIdeas`: Content ideas generated from clusters
#### Writer Module
**Purpose**: Content generation, image generation, and publishing
**Features**:
- Task creation from content ideas
- AI-powered content generation
- Content editing and review
- Image prompt extraction
- AI-powered image generation
- WordPress publishing
**Models**:
- `Tasks`: Content generation tasks
- `Content`: Generated HTML content
- `Images`: Generated images (featured, in-article)
#### System Module
**Purpose**: System configuration and AI settings
**Features**:
- AI prompt management
- Integration settings (OpenAI, Runware)
- Author profile management
- Content strategy management
- System status and monitoring
**Models**:
- `AIPrompt`: AI prompt templates
- `IntegrationSettings`: API keys and configuration
- `AuthorProfile`: Writing style profiles
- `Strategy`: Content strategies
#### Billing Module
**Purpose**: Credit management and usage tracking
**Features**:
- Credit balance tracking
- Credit transactions
- Usage logging
- Cost tracking
**Models**:
- `CreditTransaction`: Credit additions and deductions
- `CreditUsageLog`: Usage tracking with cost
#### Auth Module
**Purpose**: Multi-tenancy and user management
**Features**:
- Account management
- User management
- Plan management
- Site and sector management
- Industry templates
**Models**:
- `Account`: Top-level organization
- `User`: User accounts
- `Plan`: Subscription plans
- `Site`: Sites within accounts
- `Sector`: Sectors within sites
- `Industry`: Global industry templates
- `IndustrySector`: Industry sector templates
---
## Complete Workflows
### 1. Account Setup Workflow
**Purpose**: Onboard a new account and user
**Steps**:
1. User signs up via `/signup`
2. Account created with default plan
3. Owner user created and linked to account
4. User signs in via `/signin`
5. JWT token generated and returned
6. Frontend stores token and redirects to dashboard
7. User creates first site (optional)
8. User creates sectors (1-5 per site, optional)
9. User configures integration settings (OpenAI, Runware)
10. System ready for use
**Data Created**:
- 1 Account record
- 1 User record (owner role)
- 1 Subscription record (default plan)
- 0-N Site records
- 0-N Sector records (per site)
- 1 IntegrationSettings record (per integration type)
### 2. Keyword Management Workflow
**Purpose**: Import, organize, and cluster keywords
**Steps**:
1. User navigates to `/planner/keywords`
2. User imports keywords via CSV or manual entry
3. Keywords validated and stored in database
4. Keywords displayed in table with filters
5. User filters keywords by sector, status, intent, etc.
6. User selects keywords for clustering
7. User clicks "Auto Cluster" action
8. Backend validates keyword IDs
9. Celery task queued (`auto_cluster_keywords_task`)
10. Task ID returned to frontend
11. Frontend polls progress endpoint
12. Celery worker processes task:
- Loads keywords from database
- Builds AI prompt with keyword data
- Calls OpenAI API for clustering
- Parses cluster response
- Creates Cluster records
- Links keywords to clusters
13. Progress updates sent to frontend
14. Task completes
15. Frontend displays new clusters
16. User can view clusters, edit cluster names, add/remove keywords
**Data Created**:
- N Keyword records (if imported)
- M Cluster records (created by AI)
- N-M Keyword-to-Cluster relationships
**AI Function**: Auto Cluster Keywords
### 3. Content Idea Generation Workflow
**Purpose**: Generate content ideas from keyword clusters
**Steps**:
1. User navigates to `/planner/clusters`
2. User selects one or more clusters
3. User clicks "Generate Ideas" action
4. Backend validates cluster IDs
5. Celery task queued (`auto_generate_ideas_task`)
6. Task ID returned to frontend
7. Frontend polls progress endpoint
8. Celery worker processes task:
- Loads clusters and keywords
- Builds AI prompt with cluster data
- Calls OpenAI API for idea generation
- Parses idea response
- Creates ContentIdeas records
- Links ideas to clusters
9. Progress updates sent to frontend
10. Task completes
11. Frontend displays new content ideas
12. User can view ideas, edit titles, prioritize ideas
**Data Created**:
- N ContentIdeas records (created by AI)
- N ContentIdeas-to-Cluster relationships
**AI Function**: Generate Ideas
### 4. Content Generation Workflow
**Purpose**: Generate AI-powered content from content ideas
**Steps**:
1. User navigates to `/planner/ideas`
2. User selects one or more content ideas
3. User clicks "Create Tasks" action
4. Task records created for each idea
5. User navigates to `/writer/tasks`
6. User selects tasks for content generation
7. User clicks "Generate Content" action
8. Backend validates task IDs
9. Celery task queued (`auto_generate_content_task`)
10. Task ID returned to frontend
11. Frontend polls progress endpoint
12. Celery worker processes task:
- Loads tasks and related data (cluster, keywords, idea)
- Builds AI prompt with task data
- Calls OpenAI API for content generation
- Parses HTML content response
- Creates/updates Content records
- Updates task status
13. Progress updates sent to frontend
14. Task completes
15. Frontend displays generated content
16. User can review and edit content
17. User navigates to `/writer/content` to view content
**Data Created**:
- N Task records (from ideas)
- N Content records (generated HTML)
**AI Function**: Generate Content
### 5. Image Generation Workflow
**Purpose**: Generate images for content
**Steps**:
1. User navigates to `/writer/content`
2. User selects content items
3. User clicks "Generate Image Prompts" action (optional, can skip)
4. Backend validates content IDs
5. Celery task queued (`auto_generate_image_prompts_task`)
6. Task processes:
- Loads content HTML
- Builds AI prompt for prompt extraction
- Calls OpenAI API
- Parses image prompts (featured, in-article)
- Creates/updates Images records with prompts
7. User clicks "Generate Images" action
8. Backend validates image IDs
9. Celery task queued (`auto_generate_images_task`)
10. Task processes:
- Loads images with prompts
- Builds image generation prompt
- Calls OpenAI DALL-E or Runware API
- Receives image URLs
- Updates Images records with URLs
11. Images displayed in frontend
12. User can view images, regenerate, or delete
**Data Created**:
- N Images records (with prompts)
- N Images records (with image URLs)
**AI Functions**: Generate Image Prompts, Generate Images
### 6. WordPress Publishing Workflow
**Purpose**: Publish content to WordPress
**Steps**:
1. User navigates to `/writer/content`
2. User selects content to publish
3. User clicks "Publish to WordPress" action
4. Backend validates:
- Site has WordPress URL configured
- Site has WordPress credentials
- Content is ready (status: review or draft)
5. Backend calls WordPress REST API:
- Creates post with content HTML
- Uploads featured image (if available)
- Uploads in-article images (if available)
- Sets post status (draft, publish)
6. WordPress post ID stored in Content record
7. Content status updated to "published"
8. Frontend displays success message
9. User can view published content in WordPress
**Data Updated**:
- Content record: `wp_post_id`, `status` = "published"
**Integration**: WordPress REST API
### 7. User Management Workflow
**Purpose**: Add users to account and manage permissions
**Steps**:
1. Owner/Admin navigates to `/settings/users`
2. User clicks "Add User" action
3. User enters email and selects role
4. Backend creates User record:
- Email validated (unique per account)
- Role assigned
- Account linked
- Password set (or invitation sent)
5. If role is Editor/Viewer:
- Owner/Admin grants site access
- SiteUserAccess records created
6. New user receives invitation email
7. New user signs in and accesses granted sites
**Data Created**:
- 1 User record
- 0-N SiteUserAccess records (for Editor/Viewer roles)
### 8. Integration Configuration Workflow
**Purpose**: Configure AI service integrations
**Steps**:
1. User navigates to `/settings/integration`
2. User selects integration type (OpenAI, Runware)
3. User enters API key
4. User clicks "Test Connection" (optional)
5. Backend validates API key:
- Makes test API call
- Returns connection status
6. User saves integration settings
7. Backend stores API key in IntegrationSettings
8. Integration ready for use in AI functions
**Data Created/Updated**:
- 1 IntegrationSettings record (per integration type)
**Test Functions**: Test OpenAI, Test Runware
---
## Data Models & Relationships
### Core Models
**Account Model**:
- Fields: name, slug, owner, plan, credits, status
- Relationships: Users (1-N), Sites (1-N), Subscription (1-1)
**User Model**:
- Fields: email, account, role
- Relationships: Account (N-1), SiteUserAccess (N-M via SiteUserAccess)
**Plan Model**:
- Fields: name, price, limits (users, sites, keywords, clusters, etc.), credits
- Relationships: Accounts (1-N via Subscription)
**Site Model**:
- Fields: name, slug, domain, industry, status, wp_url, wp_username, wp_app_password
- Relationships: Account (N-1), Sectors (1-N), Industry (N-1)
**Sector Model**:
- Fields: name, slug, site, industry_sector, status
- Relationships: Site (N-1), IndustrySector (N-1), Keywords/Clusters/Ideas/Tasks (1-N)
### Planner Models
**Keywords Model**:
- Fields: keyword, volume, difficulty, intent, cluster (M-N)
- Relationships: Sector (N-1), Clusters (M-N)
**Clusters Model**:
- Fields: name, description, keywords_count, volume
- Relationships: Sector (N-1), Keywords (M-N), ContentIdeas (1-N)
**ContentIdeas Model**:
- Fields: idea_title, description, cluster, status
- Relationships: Sector (N-1), Cluster (N-1), Tasks (1-N)
### Writer Models
**Tasks Model**:
- Fields: title, description, cluster, idea, status, content
- Relationships: Sector (N-1), Cluster (N-1), ContentIdeas (N-1), Content (1-1), Images (1-N)
**Content Model**:
- Fields: task, html_content, word_count, status, wp_post_id
- Relationships: Task (1-1), Images (1-N)
**Images Model**:
- Fields: task, content, image_type, prompt, image_url, status
- Relationships: Task (N-1), Content (N-1)
### System Models
**AIPrompt Model**:
- Fields: prompt_type, prompt_value, account
- Relationships: Account (N-1)
**IntegrationSettings Model**:
- Fields: integration_type, config (JSON), account
- Relationships: Account (N-1)
**AuthorProfile Model**:
- Fields: name, description, tone, language, account
- Relationships: Account (N-1)
**Strategy Model**:
- Fields: name, description, sector, prompt_types, account
- Relationships: Account (N-1), Sector (N-1)
### Billing Models
**CreditTransaction Model**:
- Fields: account, transaction_type, amount, balance_after
- Relationships: Account (N-1)
**CreditUsageLog Model**:
- Fields: account, operation_type, credits_used, cost_usd
- Relationships: Account (N-1)
---
## Multi-Tenancy Architecture
### Account Isolation
**Implementation Levels**:
1. **Model Level**: All models inherit `AccountBaseModel` with `account` ForeignKey
2. **ViewSet Level**: All ViewSets inherit `AccountModelViewSet` with automatic filtering
3. **Middleware Level**: `AccountContextMiddleware` sets `request.account` from JWT
**Filtering**:
- All queries automatically filter by `account = request.account`
- Developer and System Bot users bypass account filtering
- System accounts bypass account filtering
### Site/Sector Hierarchy
**Purpose**: Organize content within accounts
**Structure**:
```
Account
└── Site 1 (Active)
├── Sector 1 (Active)
│ ├── Keywords
│ ├── Clusters
│ ├── ContentIdeas
│ └── Tasks
├── Sector 2 (Active)
└── Sector 3 (Inactive)
└── Site 2 (Active)
└── Sector 1 (Active)
```
**Constraints**:
- Maximum 5 active sectors per site
- Multiple sites can be active simultaneously
- All content must belong to a site and sector
### Data Isolation Flow
```
Request with JWT Token
↓
AccountContextMiddleware
├── Extract Account ID from JWT
├── Load Account Object
└── Set request.account
↓
ViewSet.get_queryset()
├── Check User Role
├── Filter by Account (if not admin/developer)
└── Filter by Accessible Sites (if not owner/admin)
↓
Database Query
↓
Results (Account-Isolated)
```
---
## API Architecture
### API Structure
**Base URL**: `/api/v1/`
**Module Endpoints**:
- `/api/v1/auth/` - Accounts, users, sites, sectors, plans
- `/api/v1/planner/` - Keywords, clusters, ideas
- `/api/v1/writer/` - Tasks, content, images
- `/api/v1/system/` - Prompts, integrations, author-profiles, strategies
- `/api/v1/billing/` - Credits, transactions, usage
### Authentication Flow
```
1. User Signs In
↓
2. Backend Validates Credentials
↓
3. JWT Token Generated
↓
4. Token Returned to Frontend
↓
5. Frontend Stores Token (localStorage)
↓
6. Frontend Includes Token in Requests (Authorization: Bearer {token})
↓
7. Backend Validates Token
↓
8. Account Context Set
↓
9. Request Processed
```
### Response Format
**Success Response**:
- `success`: true
- `data`: Response data
- `message`: Optional message
**Error Response**:
- `success`: false
- `message`: Error message
- `errors`: Validation errors (if applicable)
**Pagination Response**:
- `count`: Total count
- `next`: Next page URL
- `previous`: Previous page URL
- `results`: Array of results
---
## Security Architecture
### Authentication
**Methods**:
- JWT (JSON Web Tokens) - Primary method
- Session-based auth - Fallback for admin
**Token Management**:
- Tokens stored in localStorage
- Tokens included in `Authorization: Bearer {token}` header
- Token refresh mechanism (future implementation)
### Authorization
**Role-Based Access Control (RBAC)**:
- Role checked on every request
- Permissions enforced at ViewSet level
- Action-level permissions for sensitive operations
**Data Access Control**:
- Account-level: Automatic filtering by account
- Site-level: Filtering by accessible sites
- Action-level: Permission checks in ViewSet actions
### Data Security
**Account Isolation**:
- All queries filtered by account
- Admin/Developer override for system accounts
- No cross-account data leakage
**Site Access Control**:
- Users can only access granted sites
- Admin/Developer override for all sites
- Explicit access grants for Editor/Viewer roles
**API Security**:
- CORS configured for frontend domain
- CSRF enabled for session-based auth
- Input validation via DRF serializers
- Rate limiting (future implementation)
---
## Integration Points
### OpenAI Integration
**Purpose**: Text generation and image generation
**Configuration**:
- API key stored per account in `IntegrationSettings`
- Model selection per account
- Cost tracking per request
**Services Used**:
- GPT models for text generation
- DALL-E for image generation
### Runware Integration
**Purpose**: Alternative image generation service
**Configuration**:
- API key stored per account
- Model selection (e.g., `runware:97@1`)
- Image type selection (realistic, artistic, cartoon)
### WordPress Integration
**Purpose**: Content publishing
**Configuration**:
- WordPress URL per site
- Username and password stored per site
- REST API integration for publishing
**Workflow**:
1. Content generated in IGNY8
2. Images attached
3. Content published to WordPress via REST API
4. Status updated in IGNY8
### Stripe Integration (Planned)
**Purpose**: Payment processing
**Status**: Planned for future implementation
**Features**:
- Subscription management
- Payment processing
- Webhook integration
---
## Summary
The IGNY8 application architecture provides:
1. **Multi-Tenancy**: Complete account isolation with automatic filtering
2. **Hierarchical Organization**: Account > Site > Sector > Content structure
3. **Role-Based Access**: Granular permissions for different user roles
4. **Module-Based Design**: Clear separation of concerns across modules
5. **Complete Workflows**: End-to-end workflows from keyword import to publishing
6. **AI Integration**: Unified AI framework for all AI operations
7. **WordPress Integration**: Direct publishing to WordPress sites
8. **Credit System**: Credit-based billing and usage tracking
9. **Security First**: JWT auth, RBAC, data isolation
10. **Scalable Design**: Supports multiple accounts, sites, and users
This architecture ensures scalability, maintainability, and extensibility while providing a robust foundation for the IGNY8 platform.

479
docs/02-FRONTEND.md → docs/03-FRONTEND.md
View File

@@ -1,8 +1,7 @@
# IGNY8 Frontend Documentation # IGNY8 Frontend Documentation
**Version:** 1.0
**Last Updated:** 2025-01-XX **Last Updated:** 2025-01-XX
**Purpose:** Complete frontend documentation covering pages, components, templates, routing, state management, and configuration system. **Purpose:** Complete frontend documentation covering architecture, pages, components, routing, state management, and configuration system.
--- ---
@@ -17,9 +16,8 @@
7. [State Management](#state-management) 7. [State Management](#state-management)
8. [API Integration](#api-integration) 8. [API Integration](#api-integration)
9. [Configuration System](#configuration-system) 9. [Configuration System](#configuration-system)
10. [Pages](#pages) 10. [Pages & Features](#pages--features)
11. [Hooks](#hooks) 11. [Hooks & Utilities](#hooks--utilities)
12. [Utilities](#utilities)
--- ---
@@ -76,24 +74,58 @@ frontend/src/
│ │ ├── Dashboard.tsx │ │ ├── Dashboard.tsx
│ │ ├── Keywords.tsx │ │ ├── Keywords.tsx
│ │ ├── Clusters.tsx │ │ ├── Clusters.tsx
│ │ └── Ideas.tsx │ │ ├── Ideas.tsx
│ │ ├── KeywordOpportunities.tsx
│ │ └── Mapping.tsx
│ ├── Writer/ # Writer module pages │ ├── Writer/ # Writer module pages
│ │ ├── Dashboard.tsx │ │ ├── Dashboard.tsx
│ │ ├── Tasks.tsx │ │ ├── Tasks.tsx
│ │ ├── Content.tsx │ │ ├── Content.tsx
│ │ ├── ContentView.tsx
│ │ ├── Drafts.tsx │ │ ├── Drafts.tsx
│ │ ├── Images.tsx │ │ ├── Images.tsx
│ │ └── Published.tsx │ │ └── Published.tsx
│ ├── Thinker/ # Thinker module pages
│ │ ├── Dashboard.tsx
│ │ ├── Prompts.tsx
│ │ ├── AuthorProfiles.tsx
│ │ ├── Strategies.tsx
│ │ ├── Profile.tsx
│ │ └── ImageTesting.tsx
│ ├── Billing/ # Billing module pages
│ │ ├── Credits.tsx
│ │ ├── Transactions.tsx
│ │ └── Usage.tsx
│ ├── Settings/ # Settings pages │ ├── Settings/ # Settings pages
│ │ ├── General.tsx │ │ ├── General.tsx
│ │ ├── Users.tsx │ │ ├── Users.tsx
│ │ ├── Sites.tsx │ │ ├── Sites.tsx
│ │ ├── Integration.tsx │ │ ├── Integration.tsx
│ │ └── ... │ │ ├── AI.tsx
│ │ ├── Plans.tsx
│ │ ├── Industries.tsx
│ │ ├── Status.tsx
│ │ ├── Subscriptions.tsx
│ │ ├── Account.tsx
│ │ ├── Modules.tsx
│ │ ├── System.tsx
│ │ ├── ImportExport.tsx
│ │ └── UiElements/ # UI element examples
│ ├── Help/ # Help pages
│ │ ├── Help.tsx
│ │ ├── Docs.tsx
│ │ ├── SystemTesting.tsx
│ │ └── FunctionTesting.tsx
│ ├── Reference/ # Reference data pages
│ │ ├── Industries.tsx
│ │ └── SeedKeywords.tsx
│ ├── AuthPages/ # Authentication pages │ ├── AuthPages/ # Authentication pages
│ │ ├── SignIn.tsx │ │ ├── SignIn.tsx
│ │ └── SignUp.tsx │ │ └── SignUp.tsx
│ └── ... │ ├── Dashboard/ # Main dashboard
│ │ └── Home.tsx
│ └── OtherPage/ # Other pages
│ └── NotFound.tsx
├── templates/ # 4 universal templates ├── templates/ # 4 universal templates
│ ├── DashboardTemplate.tsx │ ├── DashboardTemplate.tsx
│ ├── TablePageTemplate.tsx │ ├── TablePageTemplate.tsx
@@ -135,8 +167,6 @@ frontend/src/
├── hooks/ # Custom React hooks ├── hooks/ # Custom React hooks
│ ├── useProgressModal.ts # Progress modal for long-running tasks │ ├── useProgressModal.ts # Progress modal for long-running tasks
│ └── useAuth.ts # Authentication hook │ └── useAuth.ts # Authentication hook
├── layout/ # Layout components
│ └── AppLayout.tsx # Main app layout wrapper
├── utils/ # Utility functions ├── utils/ # Utility functions
│ └── difficulty.ts # Difficulty utilities │ └── difficulty.ts # Difficulty utilities
├── App.tsx # Root component with routing ├── App.tsx # Root component with routing
@@ -149,54 +179,38 @@ frontend/src/
### Route Structure ### Route Structure
**File**: `App.tsx`
**Public Routes**: **Public Routes**:
- `/signin` - Sign in page - `/signin` - Sign in page
- `/signup` - Sign up page - `/signup` - Sign up page
**Protected Routes** (require authentication): **Protected Routes** (require authentication):
- `/` - Home/Dashboard - `/` - Home dashboard
- `/planner` - Planner Dashboard - `/planner/*` - Planner module routes
- `/planner/keywords` - Keywords page - `/writer/*` - Writer module routes
- `/planner/clusters` - Clusters page - `/thinker/*` - Thinker module routes
- `/planner/ideas` - Ideas page - `/billing/*` - Billing module routes
- `/writer` - Writer Dashboard - `/settings/*` - Settings routes
- `/writer/tasks` - Tasks page - `/help/*` - Help routes
- `/writer/content` - Drafts page - `/reference/*` - Reference data routes
- `/writer/images` - Images page - `/ui-elements/*` - UI element examples
- `/writer/published` - Published page
- `/thinker` - Thinker Dashboard ### ProtectedRoute Component
- `/thinker/prompts` - Prompts page
- `/thinker/author-profiles` - Author Profiles page **Purpose**: Wraps protected routes and checks authentication
- `/thinker/strategies` - Strategies page
- `/thinker/image-testing` - Image Testing page **Functionality**:
- `/billing/credits` - Credits page - Checks if user is authenticated
- `/billing/transactions` - Transactions page - Redirects to `/signin` if not authenticated
- `/billing/usage` - Usage page - Wraps children with `AppLayout` if authenticated
- `/settings` - Settings pages
- `/analytics` - Analytics page
- `/schedules` - Schedules page
### Route Configuration ### Route Configuration
**File**: `config/routes.config.ts` **File**: `config/routes.config.ts`
Defines route metadata including: **Structure**: Defines route hierarchy, labels, and icons for navigation
- Route paths
- Labels for navigation
- Icons
- Breadcrumb labels
- Nested route structure
### Protected Routes **Functions**:
- `getBreadcrumbs(pathname)`: Generates breadcrumbs for current route
**Component**: `components/auth/ProtectedRoute.tsx`
**Functionality**:
- Checks authentication status from `authStore`
- Redirects to `/signin` if not authenticated
- Wraps protected routes with `AppLayout`
--- ---
@@ -204,18 +218,7 @@ Defines route metadata including:
### 1. DashboardTemplate ### 1. DashboardTemplate
**Purpose**: Module home pages with KPIs, workflow steps, and charts. **Purpose**: Module home pages with KPIs, workflow steps, and charts
**Usage**:
```tsx
<DashboardTemplate
title="Planner Dashboard"
subtitle="Manage your SEO keywords and content planning"
metrics={metrics}
workflowSteps={steps}
charts={charts}
/>
```
**Features**: **Features**:
- Header metrics (KPIs) - Header metrics (KPIs)
@@ -223,24 +226,11 @@ Defines route metadata including:
- Charts and visualizations - Charts and visualizations
- Quick actions - Quick actions
**Usage**: Planner Dashboard, Writer Dashboard, Thinker Dashboard
### 2. TablePageTemplate ### 2. TablePageTemplate
**Purpose**: CRUD table pages (Keywords, Clusters, Tasks, etc.). **Purpose**: CRUD table pages (Keywords, Clusters, Tasks, etc.)
**Usage**:
```tsx
<TablePageTemplate
title="Keywords"
subtitle="Manage and organize SEO keywords"
columns={columns}
data={data}
filters={filterConfig}
onSort={(field, direction) => ...}
onEdit={(row) => ...}
onDelete={(row) => ...}
bulkActions={bulkActions}
/>
```
**Features**: **Features**:
- Data table with sorting - Data table with sorting
@@ -257,19 +247,11 @@ Defines route metadata including:
- `bulkActions`: Bulk action definitions - `bulkActions`: Bulk action definitions
- `actions`: Row action definitions - `actions`: Row action definitions
**Usage**: Keywords, Clusters, Ideas, Tasks, Content, Images, Prompts, etc.
### 3. FormPageTemplate ### 3. FormPageTemplate
**Purpose**: Settings/form pages (Settings, Integration, etc.). **Purpose**: Settings/form pages (Settings, Integration, etc.)
**Usage**:
```tsx
<FormPageTemplate
title="Settings"
subtitle="Configure your account settings"
sections={sections}
onSubmit={(data) => ...}
/>
```
**Features**: **Features**:
- Form sections - Form sections
@@ -277,18 +259,11 @@ Defines route metadata including:
- Save/Cancel buttons - Save/Cancel buttons
- Success/Error notifications - Success/Error notifications
**Usage**: Settings, Integration, AI Settings, Plans, etc.
### 4. SystemPageTemplate ### 4. SystemPageTemplate
**Purpose**: System/admin pages (Logs, Status, Monitoring). **Purpose**: System/admin pages (Logs, Status, Monitoring)
**Usage**:
```tsx
<SystemPageTemplate
title="System Status"
subtitle="Monitor system health and performance"
content={content}
/>
```
**Features**: **Features**:
- System information display - System information display
@@ -296,6 +271,8 @@ Defines route metadata including:
- Status indicators - Status indicators
- Performance metrics - Performance metrics
**Usage**: System Status, System Testing, etc.
--- ---
## Component Library ## Component Library
@@ -303,9 +280,7 @@ Defines route metadata including:
### Layout Components ### Layout Components
#### AppLayout #### AppLayout
**File**: `layout/AppLayout.tsx` **Purpose**: Main app layout wrapper
**Purpose**: Main app layout wrapper.
**Features**: **Features**:
- Sidebar navigation - Sidebar navigation
@@ -314,9 +289,7 @@ Defines route metadata including:
- Breadcrumbs - Breadcrumbs
#### Sidebar #### Sidebar
**File**: `components/layout/Sidebar.tsx` **Purpose**: Navigation sidebar
**Purpose**: Navigation sidebar.
**Features**: **Features**:
- Module navigation - Module navigation
@@ -324,9 +297,7 @@ Defines route metadata including:
- Collapsible sections - Collapsible sections
#### Header #### Header
**File**: `components/layout/Header.tsx` **Purpose**: Top header bar
**Purpose**: Top header bar.
**Features**: **Features**:
- User menu - User menu
@@ -336,9 +307,7 @@ Defines route metadata including:
### Table Components ### Table Components
#### DataTable #### DataTable
**File**: `components/table/DataTable.tsx` **Purpose**: Data table component
**Purpose**: Data table component.
**Features**: **Features**:
- Sortable columns - Sortable columns
@@ -347,9 +316,7 @@ Defines route metadata including:
- Responsive design - Responsive design
#### Filters #### Filters
**File**: `components/table/Filters.tsx` **Purpose**: Filter component
**Purpose**: Filter component.
**Features**: **Features**:
- Text filters - Text filters
@@ -358,9 +325,7 @@ Defines route metadata including:
- Custom filters - Custom filters
#### Pagination #### Pagination
**File**: `components/ui/pagination/CompactPagination.tsx` **Purpose**: Pagination component
**Purpose**: Pagination component.
**Features**: **Features**:
- Page navigation - Page navigation
@@ -370,52 +335,33 @@ Defines route metadata including:
### UI Components ### UI Components
#### Button #### Button
**File**: `components/ui/button/Button.tsx`
**Variants**: primary, secondary, danger, ghost, link **Variants**: primary, secondary, danger, ghost, link
**Sizes**: sm, md, lg **Sizes**: sm, md, lg
#### Card #### Card
**File**: `components/ui/card/Card.tsx` **Purpose**: Card container component
**Purpose**: Card container component.
#### Modal #### Modal
**File**: `components/ui/modal/Modal.tsx` **Purpose**: Modal dialog component
**Purpose**: Modal dialog component.
**Variants**: FormModal, ProgressModal, AlertModal **Variants**: FormModal, ProgressModal, AlertModal
#### Toast #### Toast
**File**: `components/ui/toast/ToastContainer.tsx` **Purpose**: Toast notification system
**Purpose**: Toast notification system.
**Types**: success, error, warning, info **Types**: success, error, warning, info
#### Input #### Input
**File**: `components/form/input/InputField.tsx` **Purpose**: Text input component
**Purpose**: Text input component.
#### Select #### Select
**File**: `components/form/SelectDropdown.tsx` **Purpose**: Select dropdown component
**Purpose**: Select dropdown component.
#### Checkbox #### Checkbox
**File**: `components/form/input/Checkbox.tsx` **Purpose**: Checkbox component
**Purpose**: Checkbox component.
### Auth Components ### Auth Components
#### ProtectedRoute #### ProtectedRoute
**File**: `components/auth/ProtectedRoute.tsx` **Purpose**: Route protection component
**Purpose**: Route protection component.
**Functionality**: **Functionality**:
- Checks authentication - Checks authentication
@@ -429,8 +375,6 @@ Defines route metadata including:
### Zustand Stores ### Zustand Stores
#### authStore #### authStore
**File**: `store/authStore.ts`
**State**: **State**:
- `user`: Current user object - `user`: Current user object
- `token`: JWT access token - `token`: JWT access token
@@ -448,16 +392,7 @@ Defines route metadata including:
**Persistence**: localStorage (persisted) **Persistence**: localStorage (persisted)
#### plannerStore
**File**: `store/plannerStore.ts`
**State**: Planner module-specific state
**Actions**: Planner module actions
#### siteStore #### siteStore
**File**: `store/siteStore.ts`
**State**: **State**:
- `activeSite`: Currently selected site - `activeSite`: Currently selected site
- `sites`: List of accessible sites - `sites`: List of accessible sites
@@ -469,8 +404,6 @@ Defines route metadata including:
**Persistence**: localStorage (persisted) **Persistence**: localStorage (persisted)
#### sectorStore #### sectorStore
**File**: `store/sectorStore.ts`
**State**: **State**:
- `activeSector`: Currently selected sector - `activeSector`: Currently selected sector
- `sectors`: List of sectors for active site - `sectors`: List of sectors for active site
@@ -481,9 +414,12 @@ Defines route metadata including:
**Persistence**: localStorage (persisted) **Persistence**: localStorage (persisted)
#### aiRequestLogsStore #### plannerStore
**File**: `store/aiRequestLogsStore.ts` **State**: Planner module-specific state
**Actions**: Planner module actions
#### aiRequestLogsStore
**State**: **State**:
- `logs`: Array of AI request/response logs - `logs`: Array of AI request/response logs
@@ -497,8 +433,6 @@ Defines route metadata including:
**Purpose**: Tracks AI function execution with step-by-step logs **Purpose**: Tracks AI function execution with step-by-step logs
#### pageSizeStore #### pageSizeStore
**File**: `store/pageSizeStore.ts`
**State**: **State**:
- `pageSize`: Table page size preference - `pageSize`: Table page size preference
@@ -556,29 +490,16 @@ Defines route metadata including:
**Example**: `keywords.config.tsx` **Example**: `keywords.config.tsx`
**Structure**: **Structure**: Defines columns, filters, bulkActions, actions for a page
```tsx
export const createKeywordsPageConfig = () => ({
columns: [...],
filters: [...],
bulkActions: [...],
actions: [...],
});
```
**Usage**: **Usage**: Imported in page components to configure TablePageTemplate
```tsx
import { createKeywordsPageConfig } from '../../config/pages/keywords.config';
const config = createKeywordsPageConfig();
```
### Shared Snippets ### Shared Snippets
**Location**: `config/snippets/` **Location**: `config/snippets/`
#### columns.snippets.ts #### columns.snippets.ts
**Purpose**: Reusable column definitions. **Purpose**: Reusable column definitions
**Examples**: **Examples**:
- `statusColumn`: Status column with badge - `statusColumn`: Status column with badge
@@ -586,7 +507,7 @@ const config = createKeywordsPageConfig();
- `dateColumn`: Date column with formatting - `dateColumn`: Date column with formatting
#### filters.snippets.ts #### filters.snippets.ts
**Purpose**: Reusable filter definitions. **Purpose**: Reusable filter definitions
**Examples**: **Examples**:
- `statusFilter`: Status dropdown filter - `statusFilter`: Status dropdown filter
@@ -594,7 +515,7 @@ const config = createKeywordsPageConfig();
- `searchFilter`: Text search filter - `searchFilter`: Text search filter
#### actions.snippets.ts #### actions.snippets.ts
**Purpose**: Reusable action definitions. **Purpose**: Reusable action definitions
**Examples**: **Examples**:
- `commonActions`: Edit, Delete actions - `commonActions`: Edit, Delete actions
@@ -604,34 +525,18 @@ const config = createKeywordsPageConfig();
**File**: `config/routes.config.ts` **File**: `config/routes.config.ts`
**Structure**: **Structure**: Defines route hierarchy, labels, and icons for navigation
```tsx
export const routes: RouteConfig[] = [
{
path: '/planner',
label: 'Planner',
icon: 'Planner',
children: [
{ path: '/planner/keywords', label: 'Keywords' },
...
],
},
...
];
```
**Functions**: **Functions**:
- `getBreadcrumbs(pathname)`: Get breadcrumbs for current route - `getBreadcrumbs(pathname)`: Generates breadcrumbs for current route
--- ---
## Pages ## Pages & Features
### Planner Module ### Planner Module
#### Keywords Page #### Keywords Page (`/planner/keywords`)
**File**: `pages/Planner/Keywords.tsx`
**Features**: **Features**:
- Keyword CRUD operations - Keyword CRUD operations
- Auto-cluster functionality - Auto-cluster functionality
@@ -642,28 +547,28 @@ export const routes: RouteConfig[] = [
**Configuration**: Uses `keywords.config.tsx` **Configuration**: Uses `keywords.config.tsx`
#### Clusters Page #### Clusters Page (`/planner/clusters`)
**File**: `pages/Planner/Clusters.tsx`
**Features**: **Features**:
- Cluster CRUD operations - Cluster CRUD operations
- Auto-generate ideas functionality - Auto-generate ideas functionality
- Filters (status, sector) - Filters (status, sector)
- Bulk actions - Bulk actions
#### Ideas Page #### Ideas Page (`/planner/ideas`)
**File**: `pages/Planner/Ideas.tsx`
**Features**: **Features**:
- Content ideas CRUD operations - Content ideas CRUD operations
- Filters (status, cluster, content type) - Filters (status, cluster, content type)
- Bulk actions - Bulk actions
#### Planner Dashboard (`/planner`)
**Features**:
- KPIs (total keywords, clusters, ideas)
- Workflow steps
- Charts and visualizations
### Writer Module ### Writer Module
#### Tasks Page #### Tasks Page (`/writer/tasks`)
**File**: `pages/Writer/Tasks.tsx`
**Features**: **Features**:
- Task CRUD operations - Task CRUD operations
- Auto-generate content functionality - Auto-generate content functionality
@@ -671,105 +576,145 @@ export const routes: RouteConfig[] = [
- Filters (status, cluster, content type) - Filters (status, cluster, content type)
- Bulk actions - Bulk actions
#### Content Page (`/writer/content`)
**Features**:
- Content list view
- Content detail view (`/writer/content/:id`)
- Content editing
- Generate image prompts
- Generate images
- WordPress publishing
#### Images Page (`/writer/images`)
**Features**:
- Image list view
- Image generation
- Image management
#### Writer Dashboard (`/writer`)
**Features**:
- KPIs (total tasks, content, images)
- Workflow steps
- Charts and visualizations
### Thinker Module
#### Prompts Page (`/thinker/prompts`)
**Features**:
- AI prompt CRUD operations
- Prompt type management
- Default prompt reset
#### Author Profiles Page (`/thinker/author-profiles`)
**Features**:
- Author profile CRUD operations
- Writing style configuration
#### Strategies Page (`/thinker/strategies`)
**Features**:
- Content strategy CRUD operations
- Strategy configuration
#### Image Testing Page (`/thinker/image-testing`)
**Features**:
- Image generation testing
- Prompt testing
- Model testing
### Billing Module
#### Credits Page (`/billing/credits`)
**Features**:
- Credit balance display
- Credit purchase
- Credit history
#### Transactions Page (`/billing/transactions`)
**Features**:
- Transaction history
- Transaction filtering
#### Usage Page (`/billing/usage`)
**Features**:
- Usage logs
- Cost tracking
- Usage analytics
### Settings Pages ### Settings Pages
#### Sites Page #### Sites Page (`/settings/sites`)
**File**: `pages/Settings/Sites.tsx`
**Features**: **Features**:
- Site CRUD operations - Site CRUD operations
- Site activation/deactivation - Site activation/deactivation
- Multiple sites can be active simultaneously - Multiple sites can be active simultaneously
#### Integration Page #### Integration Page (`/settings/integration`)
**File**: `pages/Settings/Integration.tsx`
**Features**: **Features**:
- Integration settings (OpenAI, Runware) - Integration settings (OpenAI, Runware)
- API key configuration - API key configuration
- Test connections - Test connections
- Image generation testing - Image generation testing
#### Users Page (`/settings/users`)
**Features**:
- User CRUD operations
- Role management
- Site access management
#### AI Settings Page (`/settings/ai`)
**Features**:
- AI prompt management
- Model configuration
--- ---
## Hooks ## Hooks & Utilities
### useProgressModal ### useProgressModal
**File**: `hooks/useProgressModal.ts` **Purpose**: Progress modal for long-running AI tasks
**Purpose**: Progress modal for long-running Celery tasks.
**Usage**:
```tsx
const progressModal = useProgressModal();
// Start task
progressModal.start(taskId, 'Task started...');
// Task progress is automatically polled
// Modal displays progress updates
```
**Features**: **Features**:
- Automatic polling of task progress - Displays progress percentage
- Progress percentage display - Shows phase messages
- Step-by-step logs (request/response steps) - Displays request/response steps
- Success/Error handling - Shows cost and token information
- Auto-close on completion - Auto-closes on completion
### useAuth ### useAuth
**File**: `hooks/useAuth.ts` **Purpose**: Authentication hook
**Purpose**: Authentication hook.
**Usage**:
```tsx
const { user, isAuthenticated, login, logout } = useAuth();
```
**Features**: **Features**:
- Access to auth state - Checks authentication status
- Login/logout functions - Provides user information
- Authentication status - Handles token refresh
--- ### Utilities
## Utilities #### difficulty.ts
**Purpose**: Difficulty calculation utilities
### Difficulty Utilities
**File**: `utils/difficulty.ts`
**Functions**: **Functions**:
- `getDifficultyLabelFromNumber(number)`: Get difficulty label (Easy, Medium, Hard) - Difficulty level calculation
- `getDifficultyRange()`: Get difficulty range options - Difficulty formatting
### API Utilities
**File**: `services/api.ts`
**Helper Functions**:
- `getActiveSiteId()`: Get active site ID from store
- `getActiveSectorId()`: Get active sector ID from store
- `getAuthToken()`: Get JWT token from store
- `getRefreshToken()`: Get refresh token from store
--- ---
## Summary ## Summary
The IGNY8 frontend is built on: The IGNY8 frontend provides:
1. **Configuration-Driven Architecture**: Zero duplication, single source of truth 1. **Configuration-Driven UI**: All pages rendered from configuration
2. **4 Universal Templates**: Reusable templates for all page types 2. **4 Universal Templates**: Reusable templates for all page types
3. **TypeScript**: Full type safety 3. **TypeScript**: Full type safety
4. **Zustand State Management**: Lightweight, performant state management 4. **Zustand State Management**: Lightweight, performant state
5. **React Router v6**: Modern routing with nested routes 5. **React Router v6**: Modern routing
6. **Component Library**: Reusable UI components 6. **Component Library**: Comprehensive UI components
7. **API Integration**: Automatic token handling and error management 7. **API Integration**: Automatic token handling and retry
8. **Responsive Design**: Mobile-first with Tailwind CSS 8. **Progress Tracking**: Real-time progress for AI tasks
9. **Responsive Design**: Mobile-first approach
10. **Complete Feature Set**: All modules and pages implemented
This architecture ensures consistency, maintainability, and extensibility while providing a great user experience. This architecture ensures consistency, maintainability, and rapid feature development while providing a great user experience.

433
docs/03-BACKEND.md → docs/04-BACKEND.md
View File

@@ -1,6 +1,5 @@
# IGNY8 Backend Documentation # IGNY8 Backend Documentation
**Version:** 1.0
**Last Updated:** 2025-01-XX **Last Updated:** 2025-01-XX
**Purpose:** Complete backend documentation covering models, views, APIs, modules, serializers, tasks, and structure. **Purpose:** Complete backend documentation covering models, views, APIs, modules, serializers, tasks, and structure.
@@ -33,7 +32,7 @@ The IGNY8 backend is a Django 5.2+ application using Django REST Framework (DRF)
- **RESTful API**: DRF ViewSets with consistent response format - **RESTful API**: DRF ViewSets with consistent response format
- **Celery Integration**: Asynchronous task processing for long-running operations - **Celery Integration**: Asynchronous task processing for long-running operations
- **Account/Site/Sector Hierarchy**: Hierarchical data organization - **Account/Site/Sector Hierarchy**: Hierarchical data organization
- **AI Integration**: Unified AIProcessor for all AI operations - **AI Integration**: Unified AI framework for all AI operations
- **Progress Tracking**: Real-time progress updates for Celery tasks - **Progress Tracking**: Real-time progress updates for Celery tasks
--- ---
@@ -75,7 +74,7 @@ backend/igny8_core/
│ │ └── urls.py # Planner module URLs │ │ └── urls.py # Planner module URLs
│ ├── writer/ # Tasks, Content, Images │ ├── writer/ # Tasks, Content, Images
│ │ ├── models.py # Tasks, Content, Images models │ │ ├── models.py # Tasks, Content, Images models
│ │ ├── views.py # TasksViewSet │ │ ├── views.py # TasksViewSet, ImagesViewSet
│ │ ├── tasks.py # Celery tasks for content/image generation │ │ ├── tasks.py # Celery tasks for content/image generation
│ │ └── urls.py # Writer module URLs │ │ └── urls.py # Writer module URLs
│ ├── system/ # Settings, Prompts, Integration │ ├── system/ # Settings, Prompts, Integration
@@ -85,18 +84,34 @@ backend/igny8_core/
│ │ ├── utils.py # Default prompts, prompt loading │ │ ├── utils.py # Default prompts, prompt loading
│ │ └── urls.py # System module URLs │ │ └── urls.py # System module URLs
│ └── billing/ # Credits, Transactions, Usage │ └── billing/ # Credits, Transactions, Usage
│ ├── models.py # CreditTransaction, UsageLog models │ ├── models.py # CreditTransaction, CreditUsageLog models
│ ├── views.py # Billing ViewSets │ ├── views.py # Billing ViewSets
│ └── services.py # CreditService │ └── services.py # CreditService
├── api/ # API base classes ├── api/ # API base classes
│ ├── base.py # AccountModelViewSet, SiteSectorModelViewSet │ ├── base.py # AccountModelViewSet, SiteSectorModelViewSet
│ └── pagination.py # CustomPageNumberPagination │ └── pagination.py # CustomPageNumberPagination
├── utils/ # Shared utilities ├── utils/ # Shared utilities
│ ├── ai_processor.py # Unified AI interface │ ├── ai_processor.py # Unified AI interface (legacy, see AI functions)
│ └── content_normalizer.py # Content processing utilities │ └── content_normalizer.py # Content processing utilities
├── middleware/ # Custom middleware ├── middleware/ # Custom middleware
│ ├── account.py # AccountContextMiddleware (sets request.account) │ ├── account.py # AccountContextMiddleware (sets request.account)
│ └── resource_tracker.py # ResourceTrackerMiddleware (API metrics) │ └── resource_tracker.py # ResourceTrackerMiddleware (API metrics)
├── ai/ # AI framework
│ ├── base.py # BaseAIFunction
│ ├── engine.py # AIEngine
│ ├── tasks.py # run_ai_task
│ ├── registry.py # Function registry
│ ├── prompts.py # PromptRegistry
│ ├── ai_core.py # AICore
│ ├── settings.py # Model configuration
│ ├── validators.py # Validation functions
│ ├── tracker.py # Progress tracking
│ └── functions/ # AI function implementations
│ ├── auto_cluster.py
│ ├── generate_ideas.py
│ ├── generate_content.py
│ ├── generate_image_prompts.py
│ └── generate_images.py
├── settings.py # Django settings ├── settings.py # Django settings
├── urls.py # Root URL configuration ├── urls.py # Root URL configuration
└── celery.py # Celery configuration └── celery.py # Celery configuration
@@ -120,7 +135,6 @@ backend/igny8_core/
**Usage**: All models that need account isolation inherit from this. **Usage**: All models that need account isolation inherit from this.
#### SiteSectorBaseModel #### SiteSectorBaseModel
**File**: `auth/models.py` **File**: `auth/models.py`
@@ -139,266 +153,207 @@ backend/igny8_core/
### Auth Models ### Auth Models
#### Account #### Account
**File**: `auth/models.py`
**Table**: `igny8_accounts` **Table**: `igny8_accounts`
**Fields**: **Fields**:
- `name`: CharField - `name`: CharField
- `slug`: SlugField (unique) - `slug`: SlugField (unique)
- `owner`: ForeignKey to User - `owner`: ForeignKey to User
- `stripe_customer_id`: CharField (optional)
- `plan`: ForeignKey to Plan - `plan`: ForeignKey to Plan
- `credits`: IntegerField (default: 0) - `credits`: IntegerField (default: 0)
- `status`: CharField (choices: active, suspended, trial, cancelled) - `status`: CharField (choices: active, suspended, trial, cancelled)
**Methods**: **Methods**:
- `is_system_account()`: Returns True if account is system account (aws-admin, default-account, default) - `is_system_account()`: Returns True if account is a system account
#### User #### User
**File**: `auth/models.py`
**Table**: `igny8_users` **Table**: `igny8_users`
**Inherits**: `AbstractUser`
**Fields**: **Fields**:
- Inherits from `AbstractUser` - `email`: EmailField (unique)
- `email`: EmailField (unique, USERNAME_FIELD)
- `account`: ForeignKey to Account - `account`: ForeignKey to Account
- `role`: CharField (choices: developer, owner, admin, editor, viewer, system_bot) - `role`: CharField (choices: developer, owner, admin, editor, viewer, system_bot)
**Methods**: **Methods**:
- `has_role(*roles)`: Check if user has any of the specified roles - `has_role(role)`: Checks if user has role
- `is_owner_or_admin()`: Returns True if role is owner or admin - `is_owner_or_admin()`: Checks if user is owner or admin
- `is_developer()`: Returns True if role is developer or is_superuser - `is_developer()`: Checks if user is developer
- `is_admin_or_developer()`: Returns True if role is admin or developer - `is_admin_or_developer()`: Checks if user is admin or developer
- `is_system_account_user()`: Returns True if user belongs to system account - `is_system_account_user()`: Checks if user belongs to system account
- `get_accessible_sites()`: Returns QuerySet of sites user can access - `get_accessible_sites()`: Returns queryset of accessible sites
#### Plan #### Plan
**File**: `auth/models.py`
**Table**: `igny8_plans` **Table**: `igny8_plans`
**Fields**: **Fields**: Extensive fields for limits (users, sites, keywords, clusters, content ideas, AI requests, word count, images, credits)
- `name`: CharField
- `slug`: SlugField (unique)
- `price`: DecimalField
- `billing_cycle`: CharField (choices: monthly, annual)
- `features`: JSONField (array of feature strings)
- `max_users`: IntegerField
- `max_sites`: IntegerField
- `max_keywords`: IntegerField
- `max_clusters`: IntegerField
- `max_content_ideas`: IntegerField
- `daily_cluster_limit`: IntegerField
- `monthly_cluster_ai_credits`: IntegerField
- `daily_content_tasks`: IntegerField
- `daily_ai_requests`: IntegerField
- `monthly_word_count_limit`: IntegerField
- `monthly_content_ai_credits`: IntegerField
- `monthly_image_count`: IntegerField
- `daily_image_generation_limit`: IntegerField
- `monthly_image_ai_credits`: IntegerField
- `max_images_per_task`: IntegerField
- `image_model_choices`: JSONField
- `included_credits`: IntegerField
- `extra_credit_price`: DecimalField
- And more...
**Methods**: **Methods**:
- `clean()`: Validates plan limits - `clean()`: Validates plan limits
- `get_effective_credits_per_month()`: Returns included_credits or credits_per_month - `get_effective_credits_per_month()`: Returns included_credits or credits_per_month
#### Site #### Site
**File**: `auth/models.py`
**Table**: `igny8_sites` **Table**: `igny8_sites`
**Inherits**: `AccountBaseModel`
**Fields**: **Fields**:
- Inherits from `AccountBaseModel`
- `name`: CharField - `name`: CharField
- `slug`: SlugField (unique per account) - `slug`: SlugField (unique per account)
- `domain`: URLField (optional) - `domain`: URLField (optional)
- `industry`: ForeignKey to Industry (optional) - `industry`: ForeignKey to Industry (optional)
- `is_active`: BooleanField - `is_active`: BooleanField
- `status`: CharField (choices: active, inactive, suspended) - `status`: CharField
- `wp_url`: URLField (optional, WordPress integration) - `wp_url`: URLField (optional)
- `wp_username`: CharField (optional) - `wp_username`: CharField (optional)
- `wp_app_password`: CharField (optional) - `wp_app_password`: CharField (optional)
**Methods**: **Methods**:
- `get_active_sectors_count()`: Get count of active sectors - `get_active_sectors_count()`: Returns count of active sectors
- `can_add_sector()`: Check if site can add another sector (max 5) - `can_add_sector()`: Returns True if site can add another sector (max 5)
#### Sector #### Sector
**File**: `auth/models.py`
**Table**: `igny8_sectors` **Table**: `igny8_sectors`
**Inherits**: `AccountBaseModel`
**Fields**: **Fields**:
- Inherits from `AccountBaseModel`
- `site`: ForeignKey to Site - `site`: ForeignKey to Site
- `industry_sector`: ForeignKey to IndustrySector (optional, template reference) - `industry_sector`: ForeignKey to IndustrySector (optional)
- `name`: CharField - `name`: CharField
- `slug`: SlugField (unique per site) - `slug`: SlugField (unique per site)
- `is_active`: BooleanField - `is_active`: BooleanField
- `status`: CharField (choices: active, inactive) - `status`: CharField
**Validation**: Maximum 5 active sectors per site
### Planner Models ### Planner Models
#### Keywords #### Keywords
**File**: `modules/planner/models.py`
**Table**: `igny8_keywords` **Table**: `igny8_keywords`
**Inherits**: `SiteSectorBaseModel`
**Fields**: **Fields**:
- Inherits from `SiteSectorBaseModel`
- `keyword`: CharField - `keyword`: CharField
- `volume`: IntegerField - `volume`: IntegerField (optional)
- `difficulty`: IntegerField - `difficulty`: IntegerField (optional)
- `intent`: CharField (choices: informational, navigational, commercial, transactional) - `intent`: CharField (optional)
- `cluster`: ForeignKey to Clusters (optional) - `status`: CharField
- `status`: CharField (choices: active, pending, archived) - `cluster`: ManyToManyField to Clusters
#### Clusters #### Clusters
**File**: `modules/planner/models.py`
**Table**: `igny8_clusters` **Table**: `igny8_clusters`
**Inherits**: `SiteSectorBaseModel`
**Fields**: **Fields**:
- Inherits from `SiteSectorBaseModel` - `name`: CharField
- `name`: CharField (unique) - `description`: TextField (optional)
- `description`: TextField - `keywords_count`: IntegerField (calculated)
- `keywords_count`: IntegerField - `volume`: IntegerField (calculated)
- `volume`: IntegerField
- `mapped_pages`: IntegerField
- `status`: CharField - `status`: CharField
- `keywords`: ManyToManyField to Keywords
#### ContentIdeas #### ContentIdeas
**File**: `modules/planner/models.py`
**Table**: `igny8_content_ideas` **Table**: `igny8_content_ideas`
**Inherits**: `SiteSectorBaseModel`
**Fields**: **Fields**:
- Inherits from `SiteSectorBaseModel`
- `idea_title`: CharField - `idea_title`: CharField
- `description`: TextField - `description`: TextField
- `content_structure`: CharField (choices: cluster_hub, landing_page, pillar_page, supporting_page) - `content_type`: CharField
- `content_type`: CharField (choices: blog_post, article, guide, tutorial) - `content_structure`: CharField
- `target_keywords`: CharField (comma-separated, legacy) - `target_keywords`: TextField
- `keyword_objects`: ManyToManyField to Keywords
- `keyword_cluster`: ForeignKey to Clusters - `keyword_cluster`: ForeignKey to Clusters
- `status`: CharField (choices: new, scheduled, published)
- `estimated_word_count`: IntegerField - `estimated_word_count`: IntegerField
- `status`: CharField
### Writer Models ### Writer Models
#### Tasks #### Tasks
**File**: `modules/writer/models.py`
**Table**: `igny8_tasks` **Table**: `igny8_tasks`
**Inherits**: `SiteSectorBaseModel`
**Fields**: **Fields**:
- Inherits from `SiteSectorBaseModel`
- `title`: CharField - `title`: CharField
- `description`: TextField - `description`: TextField
- `keywords`: CharField (comma-separated, legacy) - `cluster`: ForeignKey to Clusters (optional)
- `keyword_objects`: ManyToManyField to Keywords - `idea`: ForeignKey to ContentIdeas (optional)
- `cluster`: ForeignKey to Clusters
- `idea`: ForeignKey to ContentIdeas
- `content_structure`: CharField
- `content_type`: CharField - `content_type`: CharField
- `status`: CharField (choices: queued, in_progress, draft, review, published, completed) - `content_structure`: CharField
- `content`: TextField (generated content) - `status`: CharField
- `word_count`: IntegerField
- `meta_title`: CharField
- `meta_description`: TextField
- `assigned_post_id`: IntegerField (WordPress post ID)
- `post_url`: URLField
#### Content #### Content
**File**: `modules/writer/models.py`
**Table**: `igny8_content` **Table**: `igny8_content`
**Inherits**: `SiteSectorBaseModel`
**Fields**: **Fields**:
- Inherits from `SiteSectorBaseModel`
- `task`: OneToOneField to Tasks - `task`: OneToOneField to Tasks
- `html_content`: TextField - `html_content`: TextField
- `word_count`: IntegerField - `word_count`: IntegerField
- `metadata`: JSONField - `status`: CharField
- `wp_post_id`: IntegerField (optional)
**Methods**: - `meta_title`: CharField (optional)
- `save()`: Automatically sets account, site, sector from task - `meta_description`: TextField (optional)
- `primary_keyword`: CharField (optional)
- `secondary_keywords`: TextField (optional)
#### Images #### Images
**File**: `modules/writer/models.py`
**Table**: `igny8_images` **Table**: `igny8_images`
**Fields**: **Inherits**: `SiteSectorBaseModel`
- Inherits from `SiteSectorBaseModel`
- `task`: ForeignKey to Tasks
- `image_type`: CharField (choices: featured, desktop, mobile, in_article)
- `image_url`: URLField
- `image_path`: CharField (local path)
- `prompt`: TextField
- `status`: CharField
- `position`: IntegerField
**Methods**: **Fields**:
- `save()`: Automatically sets account, site, sector from task - `task`: ForeignKey to Tasks (optional)
- `content`: ForeignKey to Content (optional)
- `image_type`: CharField (choices: featured, in_article, desktop, mobile)
- `prompt`: TextField
- `image_url`: CharField
- `status`: CharField
### System Models ### System Models
#### AIPrompt #### AIPrompt
**File**: `modules/system/models.py`
**Table**: `igny8_ai_prompts` **Table**: `igny8_ai_prompts`
**Fields**: **Inherits**: `AccountBaseModel`
- Inherits from `AccountBaseModel`
- `prompt_type`: CharField (choices: clustering, ideas, content_generation, image_prompt_extraction, image_prompt_template, negative_prompt)
- `prompt_value`: TextField
- `default_prompt`: TextField
- `is_active`: BooleanField
**Unique Constraint**: `(account, prompt_type)` **Fields**:
- `prompt_type`: CharField
- `prompt_value`: TextField
- `function_name`: CharField (optional)
#### IntegrationSettings #### IntegrationSettings
**File**: `modules/system/models.py`
**Table**: `igny8_integration_settings` **Table**: `igny8_integration_settings`
**Fields**: **Inherits**: `AccountBaseModel`
- Inherits from `AccountBaseModel`
- `integration_type`: CharField (choices: openai, runware, gsc, image_generation)
- `config`: JSONField (API keys, settings, etc.)
- `is_active`: BooleanField
**Unique Constraint**: `(account, integration_type)` **Fields**:
- `integration_type`: CharField (choices: openai, runware, image_generation)
- `config`: JSONField
#### AuthorProfile #### AuthorProfile
**File**: `modules/system/models.py`
**Table**: `igny8_author_profiles` **Table**: `igny8_author_profiles`
**Inherits**: `AccountBaseModel`
**Fields**: **Fields**:
- Inherits from `AccountBaseModel`
- `name`: CharField - `name`: CharField
- `description`: TextField - `description`: TextField
- `tone`: CharField - `tone`: CharField
- `language`: CharField - `language`: CharField
- `structure_template`: JSONField
- `is_active`: BooleanField
#### Strategy #### Strategy
**File**: `modules/system/models.py`
**Table**: `igny8_strategies` **Table**: `igny8_strategies`
**Inherits**: `AccountBaseModel`
**Fields**: **Fields**:
- Inherits from `AccountBaseModel`
- `name`: CharField - `name`: CharField
- `description`: TextField - `description`: TextField
- `sector`: ForeignKey to Sector (optional) - `sector`: ForeignKey to Sector (optional)
@@ -406,6 +361,30 @@ backend/igny8_core/
- `section_logic`: JSONField - `section_logic`: JSONField
- `is_active`: BooleanField - `is_active`: BooleanField
### Billing Models
#### CreditTransaction
**Table**: `igny8_credit_transactions`
**Inherits**: `AccountBaseModel`
**Fields**:
- `transaction_type`: CharField (choices: purchase, usage, refund, adjustment)
- `amount`: IntegerField
- `balance_after`: IntegerField
- `description`: TextField (optional)
#### CreditUsageLog
**Table**: `igny8_credit_usage_logs`
**Inherits**: `AccountBaseModel`
**Fields**:
- `operation_type`: CharField
- `credits_used`: IntegerField
- `cost_usd`: DecimalField
- `details`: JSONField (optional)
--- ---
## ViewSets ## ViewSets
@@ -448,8 +427,6 @@ backend/igny8_core/
### Planner ViewSets ### Planner ViewSets
#### KeywordViewSet #### KeywordViewSet
**File**: `modules/planner/views.py`
**Inherits**: `SiteSectorModelViewSet` **Inherits**: `SiteSectorModelViewSet`
**Actions**: **Actions**:
@@ -471,8 +448,6 @@ backend/igny8_core/
- Ordering: `created_at`, `volume`, `difficulty` - Ordering: `created_at`, `volume`, `difficulty`
#### ClusterViewSet #### ClusterViewSet
**File**: `modules/planner/views.py`
**Inherits**: `SiteSectorModelViewSet` **Inherits**: `SiteSectorModelViewSet`
**Actions**: **Actions**:
@@ -484,8 +459,6 @@ backend/igny8_core/
- `auto_generate_ideas()`: Auto-generate content ideas for clusters - `auto_generate_ideas()`: Auto-generate content ideas for clusters
#### ContentIdeasViewSet #### ContentIdeasViewSet
**File**: `modules/planner/views.py`
**Inherits**: `SiteSectorModelViewSet` **Inherits**: `SiteSectorModelViewSet`
**Actions**: **Actions**:
@@ -498,8 +471,6 @@ backend/igny8_core/
### Writer ViewSets ### Writer ViewSets
#### TasksViewSet #### TasksViewSet
**File**: `modules/writer/views.py`
**Inherits**: `SiteSectorModelViewSet` **Inherits**: `SiteSectorModelViewSet`
**Actions**: **Actions**:
@@ -509,7 +480,7 @@ backend/igny8_core/
- `update()`: Update task - `update()`: Update task
- `destroy()`: Delete task - `destroy()`: Delete task
- `auto_generate_content()`: Auto-generate content for tasks - `auto_generate_content()`: Auto-generate content for tasks
- `auto_generate_images()`: Auto-generate images for tasks - `generate_images()`: Generate images for image records
- `bulk_delete()`: Bulk delete tasks - `bulk_delete()`: Bulk delete tasks
- `bulk_update()`: Bulk update task status - `bulk_update()`: Bulk update task status
@@ -518,11 +489,20 @@ backend/igny8_core/
- Filters: `status`, `cluster_id`, `content_type`, `content_structure` - Filters: `status`, `cluster_id`, `content_type`, `content_structure`
- Ordering: `title`, `created_at`, `word_count`, `status` - Ordering: `title`, `created_at`, `word_count`, `status`
#### ImagesViewSet
**Inherits**: `SiteSectorModelViewSet`
**Actions**:
- `list()`: List images
- `create()`: Create image
- `retrieve()`: Get image details
- `update()`: Update image
- `destroy()`: Delete image
- `generate_images()`: Generate images using AI
### System ViewSets ### System ViewSets
#### IntegrationSettingsViewSet #### IntegrationSettingsViewSet
**File**: `modules/system/integration_views.py`
**Inherits**: `viewsets.ViewSet` **Inherits**: `viewsets.ViewSet`
**Actions**: **Actions**:
@@ -537,8 +517,6 @@ backend/igny8_core/
- `task_progress()`: Get Celery task progress - `task_progress()`: Get Celery task progress
#### AIPromptViewSet #### AIPromptViewSet
**File**: `modules/system/views.py`
**Inherits**: `AccountModelViewSet` **Inherits**: `AccountModelViewSet`
**Actions**: **Actions**:
@@ -550,8 +528,6 @@ backend/igny8_core/
- `reset_to_default()`: Reset prompt to default value - `reset_to_default()`: Reset prompt to default value
#### AuthorProfileViewSet #### AuthorProfileViewSet
**File**: `modules/system/views.py`
**Inherits**: `AccountModelViewSet` **Inherits**: `AccountModelViewSet`
**Actions**: **Actions**:
@@ -568,63 +544,65 @@ backend/igny8_core/
### Planner Serializers ### Planner Serializers
#### KeywordSerializer #### KeywordSerializer
**File**: `modules/planner/serializers.py`
**Fields**: All Keyword model fields **Fields**: All Keyword model fields
**Validation**: Validates keyword uniqueness, cluster belongs to same sector **Validation**: Validates keyword uniqueness, cluster belongs to same sector
#### ClusterSerializer #### ClusterSerializer
**File**: `modules/planner/cluster_serializers.py`
**Fields**: All Cluster model fields **Fields**: All Cluster model fields
**Read-Only Fields**: `keywords_count`, `volume` (calculated) **Read-Only Fields**: `keywords_count`, `volume` (calculated)
#### ContentIdeasSerializer #### ContentIdeasSerializer
**File**: `modules/planner/serializers.py`
**Fields**: All ContentIdeas model fields **Fields**: All ContentIdeas model fields
### Writer Serializers ### Writer Serializers
#### TasksSerializer #### TasksSerializer
**File**: `modules/writer/serializers.py`
**Fields**: All Tasks model fields **Fields**: All Tasks model fields
#### ContentSerializer #### ContentSerializer
**File**: `modules/writer/serializers.py`
**Fields**: All Content model fields **Fields**: All Content model fields
#### ImagesSerializer #### ImagesSerializer
**File**: `modules/writer/serializers.py`
**Fields**: All Images model fields **Fields**: All Images model fields
### System Serializers ### System Serializers
#### AIPromptSerializer #### AIPromptSerializer
**File**: `modules/system/serializers.py`
**Fields**: All AIPrompt model fields **Fields**: All AIPrompt model fields
#### IntegrationSettingsSerializer #### IntegrationSettingsSerializer
**File**: `modules/system/serializers.py`
**Fields**: All IntegrationSettings model fields **Fields**: All IntegrationSettings model fields
--- ---
## Celery Tasks ## Celery Tasks
### AI Task Entry Point
#### run_ai_task
**File**: `ai/tasks.py`
**Purpose**: Unified Celery task entrypoint for all AI functions
**Parameters**:
- `function_name`: Function name (e.g., 'auto_cluster')
- `payload`: Function-specific payload
- `account_id`: Account ID
**Flow**:
1. Gets account from account_id
2. Gets function instance from registry
3. Creates AIEngine
4. Executes function via AIEngine
### Planner Tasks ### Planner Tasks
#### auto_cluster_keywords_task #### auto_cluster_keywords_task
**File**: `modules/planner/tasks.py` **File**: `modules/planner/tasks.py` (legacy, now uses `run_ai_task`)
**Purpose**: Auto-cluster keywords using AI. **Purpose**: Auto-cluster keywords using AI
**Parameters**: **Parameters**:
- `keyword_ids`: List of keyword IDs - `keyword_ids`: List of keyword IDs
@@ -634,12 +612,10 @@ backend/igny8_core/
**Progress Tracking**: Updates progress with request_steps and response_steps **Progress Tracking**: Updates progress with request_steps and response_steps
**Calls**: `_auto_cluster_keywords_core()`
#### auto_generate_ideas_task #### auto_generate_ideas_task
**File**: `modules/planner/tasks.py` **File**: `modules/planner/tasks.py` (legacy, now uses `run_ai_task`)
**Purpose**: Auto-generate content ideas for clusters. **Purpose**: Auto-generate content ideas for clusters
**Parameters**: **Parameters**:
- `cluster_ids`: List of cluster IDs - `cluster_ids`: List of cluster IDs
@@ -647,14 +623,12 @@ backend/igny8_core/
**Progress Tracking**: Updates progress for each cluster **Progress Tracking**: Updates progress for each cluster
**Calls**: `_generate_single_idea_core()` for each cluster
### Writer Tasks ### Writer Tasks
#### auto_generate_content_task #### auto_generate_content_task
**File**: `modules/writer/tasks.py` **File**: `modules/writer/tasks.py` (legacy, now uses `run_ai_task`)
**Purpose**: Auto-generate content for tasks. **Purpose**: Auto-generate content for tasks
**Parameters**: **Parameters**:
- `task_ids`: List of task IDs - `task_ids`: List of task IDs
@@ -662,20 +636,17 @@ backend/igny8_core/
**Progress Tracking**: Updates progress for each task **Progress Tracking**: Updates progress for each task
**Calls**: `AIProcessor.generate_content()` #### process_image_generation_queue
**File**: `modules/writer/views.py` (via `ai/tasks.py`)
#### auto_generate_images_task **Purpose**: Generate images for image records
**File**: `modules/writer/tasks.py`
**Purpose**: Auto-generate images for tasks.
**Parameters**: **Parameters**:
- `task_ids`: List of task IDs - `image_ids`: List of image IDs
- `account_id`: Account ID - `account_id`: Account ID
- `content_id`: Content ID (optional)
**Progress Tracking**: Updates progress for each task **Progress Tracking**: Updates progress for each image sequentially
**Calls**: `AIProcessor.extract_image_prompts()` and `AIProcessor.generate_image()`
--- ---
@@ -705,7 +676,7 @@ backend/igny8_core/
- `GET /api/v1/writer/tasks/` - List tasks - `GET /api/v1/writer/tasks/` - List tasks
- `POST /api/v1/writer/tasks/auto_generate_content/` - Auto-generate content - `POST /api/v1/writer/tasks/auto_generate_content/` - Auto-generate content
- `POST /api/v1/writer/tasks/auto_generate_images/` - Auto-generate images - `POST /api/v1/writer/images/generate_images/` - Generate images
### System Endpoints ### System Endpoints
@@ -724,7 +695,7 @@ backend/igny8_core/
**File**: `api/base.py` **File**: `api/base.py`
**Purpose**: Base ViewSet with automatic account filtering. **Purpose**: Base ViewSet with automatic account filtering
**Features**: **Features**:
- Automatic account filtering - Automatic account filtering
@@ -735,7 +706,7 @@ backend/igny8_core/
**File**: `api/base.py` **File**: `api/base.py`
**Purpose**: Base ViewSet with site/sector filtering. **Purpose**: Base ViewSet with site/sector filtering
**Features**: **Features**:
- Account filtering (inherited) - Account filtering (inherited)
@@ -751,7 +722,7 @@ backend/igny8_core/
**File**: `middleware/account.py` **File**: `middleware/account.py`
**Purpose**: Sets `request.account` from JWT token. **Purpose**: Sets `request.account` from JWT token
**Functionality**: **Functionality**:
- Extracts account ID from JWT token - Extracts account ID from JWT token
@@ -762,7 +733,7 @@ backend/igny8_core/
**File**: `middleware/resource_tracker.py` **File**: `middleware/resource_tracker.py`
**Purpose**: Tracks API request metrics. **Purpose**: Tracks API request metrics
**Functionality**: **Functionality**:
- Tracks CPU, memory, I/O usage - Tracks CPU, memory, I/O usage
@@ -773,29 +744,15 @@ backend/igny8_core/
## Utilities ## Utilities
### AIProcessor
**File**: `utils/ai_processor.py`
**Purpose**: Unified AI interface for all AI operations.
**Methods**:
- `cluster_keywords()`: Cluster keywords using AI
- `generate_ideas()`: Generate content ideas
- `generate_content()`: Generate text content
- `extract_image_prompts()`: Extract image prompts from content
- `generate_image()`: Generate images using OpenAI DALL-E or Runware
**See**: AI Functions documentation for complete details
### Content Normalizer ### Content Normalizer
**File**: `utils/content_normalizer.py` **File**: `utils/content_normalizer.py`
**Purpose**: Content processing utilities. **Purpose**: Content processing utilities
**Functions**: **Functions**:
- `_extract_body_content()`: Extract body content from HTML - `normalize_content()`: Converts plain text to HTML
- `_extract_body_content()`: Extracts body content from HTML
--- ---
@@ -803,58 +760,58 @@ backend/igny8_core/
### Planner Module ### Planner Module
**Purpose**: Keyword management and content planning. **Purpose**: Keyword management and content planning
**Models**: Keywords, Clusters, ContentIdeas **Models**: Keywords, Clusters, ContentIdeas
**ViewSets**: KeywordViewSet, ClusterViewSet, ContentIdeasViewSet **ViewSets**: KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
**Tasks**: auto_cluster_keywords_task, auto_generate_ideas_task **Tasks**: Auto-cluster keywords, Auto-generate ideas
### Writer Module ### Writer Module
**Purpose**: Content generation and management. **Purpose**: Content generation and management
**Models**: Tasks, Content, Images **Models**: Tasks, Content, Images
**ViewSets**: TasksViewSet **ViewSets**: TasksViewSet, ImagesViewSet
**Tasks**: auto_generate_content_task, auto_generate_images_task **Tasks**: Auto-generate content, Generate images
### System Module ### System Module
**Purpose**: System settings, prompts, and integrations. **Purpose**: System configuration and AI settings
**Models**: AIPrompt, IntegrationSettings, AuthorProfile, Strategy **Models**: AIPrompt, IntegrationSettings, AuthorProfile, Strategy
**ViewSets**: AIPromptViewSet, AuthorProfileViewSet, IntegrationSettingsViewSet **ViewSets**: AIPromptViewSet, IntegrationSettingsViewSet, AuthorProfileViewSet
**Utilities**: Default prompts, prompt loading
### Billing Module ### Billing Module
**Purpose**: Credits, transactions, and usage tracking. **Purpose**: Credit management and usage tracking
**Models**: CreditTransaction, UsageLog **Models**: CreditTransaction, CreditUsageLog
**ViewSets**: CreditTransactionViewSet, UsageLogViewSet **ViewSets**: CreditTransactionViewSet, CreditUsageLogViewSet
**Services**: CreditService **Services**: CreditService (check, deduct, add, calculate credits)
--- ---
## Summary ## Summary
The IGNY8 backend is built on: The IGNY8 backend provides:
1. **Django + DRF**: Robust web framework with RESTful API 1. **Multi-Tenancy**: Complete account isolation with automatic filtering
2. **Multi-Tenancy**: Complete account isolation with automatic filtering 2. **RESTful API**: DRF ViewSets with consistent response format
3. **Modular Architecture**: Clear module boundaries with shared utilities 3. **Celery Integration**: Asynchronous task processing
4. **Celery Integration**: Asynchronous task processing for long-running operations 4. **Hierarchical Organization**: Account > Site > Sector > Content structure
5. **Base ViewSets**: Consistent access control and filtering 5. **AI Framework**: Unified AI framework for all AI operations
6. **AI Integration**: Unified AIProcessor for all AI operations 6. **Progress Tracking**: Real-time progress updates for Celery tasks
7. **Progress Tracking**: Real-time progress updates for Celery tasks 7. **Module-Based Design**: Clear separation of concerns
8. **Account/Site/Sector Hierarchy**: Hierarchical data organization 8. **Base Classes**: Reusable ViewSets for common patterns
9. **Middleware**: Account context and resource tracking
10. **Utilities**: Content processing and AI integration
This architecture ensures scalability, maintainability, and extensibility while providing a robust API for the frontend. This architecture ensures scalability, maintainability, and extensibility while providing a robust foundation for the IGNY8 platform.

739
docs/05-ACCOUNT-USER-PLAN.md
View File

@@ -1,739 +0,0 @@
# IGNY8 Account, User, Plan, Sites, Credits Documentation
**Version:** 1.0
**Last Updated:** 2025-01-XX
**Purpose:** Complete documentation of multi-tenancy architecture, access control, plans, credits, and related systems.
---
## Table of Contents
1. [Overview](#overview)
2. [Account Model](#account-model)
3. [User Model](#user-model)
4. [Plan Model](#plan-model)
5. [Subscription Model](#subscription-model)
6. [Site Model](#site-model)
7. [Sector Model](#sector-model)
8. [Industry & IndustrySector Models](#industry--industrysector-models)
9. [Access Control](#access-control)
10. [Credits System](#credits-system)
11. [Plan Limits](#plan-limits)
12. [Multi-Tenancy Architecture](#multi-tenancy-architecture)
---
## Overview
The IGNY8 platform uses a multi-account architecture with the following core entities:
- **Account**: Top-level organization/workspace
- **User**: Individual user accounts with roles
- **Plan**: Subscription plan templates with limits
- **Subscription**: Active subscription linking Account to Plan
- **Site**: Workspace within an Account (1-N relationship)
- **Sector**: Content category within a Site (1-5 per site)
- **Industry**: Global industry templates
- **IndustrySector**: Industry sector templates
### Key Relationships
```
Account (1) ──< (N) User
Account (1) ──< (1) Subscription ──> (1) Plan
Account (1) ──< (N) Site
Site (1) ──< (1-5) Sector
Industry (1) ──< (N) IndustrySector
Site (1) ──> (1) Industry (optional)
Sector (1) ──> (1) IndustrySector (optional, template reference)
```
---
## Account Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_accounts`
### Fields
- `id`: Primary key
- `name`: CharField - Account name
- `slug`: SlugField - Unique slug identifier
- `owner`: ForeignKey to User - Account owner
- `stripe_customer_id`: CharField - Stripe customer ID (optional)
- `plan`: ForeignKey to Plan - Current subscription plan
- `credits`: IntegerField - Current credit balance (default: 0)
- `status`: CharField - Account status (choices: active, suspended, trial, cancelled)
- `created_at`: DateTimeField - Creation timestamp
- `updated_at`: DateTimeField - Last update timestamp
### Status Values
- `active`: Normal operation
- `suspended`: Access temporarily revoked
- `trial`: Limited trial access
- `cancelled`: Account terminated
### Methods
#### is_system_account()
**Returns**: `True` if account is a system account (aws-admin, default-account, default)
**Purpose**: System accounts bypass all filtering restrictions
### Relationships
- **One-to-Many**: Users (via `user.account`)
- **One-to-One**: Subscription (via `account.subscription`)
- **One-to-Many**: Sites (via `site.account`)
---
## User Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_users`
**Inherits**: `AbstractUser` (Django's built-in user model)
### Fields
- `id`: Primary key (inherited from AbstractUser)
- `email`: EmailField - Unique, USERNAME_FIELD
- `username`: CharField - Username
- `password`: Hashed password (inherited)
- `account`: ForeignKey to Account - User's account (null=True, blank=True)
- `role`: CharField - User role (choices: developer, owner, admin, editor, viewer, system_bot)
- `created_at`: DateTimeField - Creation timestamp
- `updated_at`: DateTimeField - Last update timestamp
### Role Hierarchy (from highest to lowest)
1. **developer**: Full system access, bypasses all restrictions
2. **owner**: Full access to account, can manage users and billing
3. **admin**: Admin access to account, can manage content and users
4. **editor**: Can edit content, manage clusters/tasks
5. **viewer**: Read-only access
6. **system_bot**: System automation user
### Methods
#### has_role(*roles)
**Parameters**: `*roles` - Variable number of role strings
**Returns**: `True` if user has any of the specified roles
#### is_owner_or_admin()
**Returns**: `True` if role is owner or admin
#### is_developer()
**Returns**: `True` if role is developer or is_superuser
#### is_admin_or_developer()
**Returns**: `True` if role is admin or developer (bypasses restrictions)
**Purpose**: Admin/Developer users bypass account/site/sector filtering
#### is_system_account_user()
**Returns**: `True` if user belongs to system account
**Purpose**: System account users bypass all filtering restrictions
#### get_accessible_sites()
**Returns**: QuerySet of sites user can access
**Access Control Logic**:
- **System Account Users**: All active sites across all accounts
- **Developers**: All active sites across all accounts
- **Owners/Admins**: All sites in their account
- **Editors/Viewers**: Only sites explicitly granted via `SiteUserAccess`
### Relationships
- **Many-to-One**: Account (via `user.account`)
- **Many-to-Many**: Sites (via `SiteUserAccess`)
---
## Plan Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_plans`
### Fields
#### Plan Information
- `id`: Primary key
- `name`: CharField - Plan name (e.g., "Free", "Starter", "Growth")
- `slug`: SlugField - Unique slug identifier
- `price`: DecimalField - Monthly price
- `billing_cycle`: CharField - Billing cycle (choices: monthly, annual)
- `features`: JSONField - Array of feature strings (e.g., ['ai_writer', 'image_gen', 'auto_publish'])
- `is_active`: BooleanField - Whether plan is available for subscription
#### User/Site/Scope Limits
- `max_users`: IntegerField - Total users allowed per account (default: 1, min: 1)
- `max_sites`: IntegerField - Maximum sites allowed (default: 1, min: 1)
- `max_industries`: IntegerField - Optional limit for industries/sectors (nullable)
- `max_author_profiles`: IntegerField - Limit for saved writing styles (default: 5)
#### Planner Limits
- `max_keywords`: IntegerField - Total keywords allowed globally (default: 1000)
- `max_clusters`: IntegerField - Total clusters allowed globally (default: 100)
- `max_content_ideas`: IntegerField - Total content ideas allowed globally (default: 300)
- `daily_cluster_limit`: IntegerField - Max clusters per day (default: 10)
- `daily_keyword_import_limit`: IntegerField - Seed keywords import limit per day (default: 100)
- `monthly_cluster_ai_credits`: IntegerField - AI credits for clustering (default: 50)
#### Writer Limits
- `daily_content_tasks`: IntegerField - Max content tasks per day (default: 10)
- `daily_ai_requests`: IntegerField - Total AI executions per day (default: 50)
- `monthly_word_count_limit`: IntegerField - Monthly word limit for generated content (default: 50000)
- `monthly_content_ai_credits`: IntegerField - AI credit pool for content generation (default: 200)
#### Image Generation Limits
- `monthly_image_count`: IntegerField - Max images per month (default: 100)
- `daily_image_generation_limit`: IntegerField - Max images per day (default: 25)
- `monthly_image_ai_credits`: IntegerField - AI credit pool for images (default: 100)
- `max_images_per_task`: IntegerField - Max images per content task (default: 4, min: 1)
- `image_model_choices`: JSONField - Allowed image models (e.g., ['dalle3', 'hidream'])
#### AI Request Controls
- `daily_ai_request_limit`: IntegerField - Global daily AI request cap (default: 100)
- `monthly_ai_credit_limit`: IntegerField - Unified credit ceiling per month (default: 500)
#### Billing & Credits
- `included_credits`: IntegerField - Monthly credits included (default: 0)
- `extra_credit_price`: DecimalField - Price per additional credit (default: 0.01)
- `allow_credit_topup`: BooleanField - Can user purchase more credits? (default: True)
- `auto_credit_topup_threshold`: IntegerField - Auto top-up trigger point (nullable)
- `auto_credit_topup_amount`: IntegerField - Credits to auto-buy (nullable)
#### Stripe Integration
- `stripe_product_id`: CharField - Stripe product ID (nullable)
- `stripe_price_id`: CharField - Stripe price ID (nullable)
#### Legacy Fields
- `credits_per_month`: IntegerField - DEPRECATED: Use included_credits instead (default: 0)
### Methods
#### clean()
**Purpose**: Validates plan limits
**Validations**:
- `max_sites` must be >= 1
- `included_credits` must be >= 0
#### get_effective_credits_per_month()
**Returns**: `included_credits` if set, otherwise `credits_per_month` (backward compatibility)
### Relationships
- **One-to-Many**: Accounts (via `account.plan`)
### Plan Limits Enforcement
**Current Status**: Plan limits are defined but **not currently enforced** in AI functions.
**Future Implementation**: Plan limits should be checked before allowing operations.
---
## Subscription Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_subscriptions`
### Fields
- `id`: Primary key
- `account`: OneToOneField to Account - Account subscription
- `stripe_subscription_id`: CharField - Unique Stripe subscription ID
- `status`: CharField - Subscription status (choices: active, past_due, canceled, trialing)
- `current_period_start`: DateTimeField - Current billing period start
- `current_period_end`: DateTimeField - Current billing period end
- `cancel_at_period_end`: BooleanField - Cancel at period end flag (default: False)
- `created_at`: DateTimeField - Creation timestamp
- `updated_at`: DateTimeField - Last update timestamp
### Status Values
- `active`: Subscription is active and paid
- `past_due`: Payment failed, subscription past due
- `canceled`: Subscription canceled
- `trialing`: In trial period
### Methods
### Relationships
- **One-to-One**: Account (via `account.subscription`)
---
## Site Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_sites`
**Inherits**: `AccountBaseModel`
### Fields
- `id`: Primary key
- `account`: ForeignKey to Account - Account this site belongs to
- `name`: CharField - Site name
- `slug`: SlugField - Unique slug per account
- `domain`: URLField - Primary domain URL (optional)
- `description`: TextField - Site description (optional)
- `industry`: ForeignKey to Industry - Industry this site belongs to (optional)
- `is_active`: BooleanField - Whether site is active (default: True)
- `status`: CharField - Site status (choices: active, inactive, suspended, default: active)
- `wp_url`: URLField - WordPress site URL (optional, for WordPress integration)
- `wp_username`: CharField - WordPress username (optional)
- `wp_app_password`: CharField - WordPress app password (optional)
- `created_at`: DateTimeField - Creation timestamp
- `updated_at`: DateTimeField - Last update timestamp
### Unique Constraint
- `(account, slug)` - Slug unique per account
### Methods
#### get_active_sectors_count()
**Returns**: Count of active sectors for this site
#### can_add_sector()
**Returns**: `True` if site can add another sector (max 5 sectors per site)
### Relationships
- **Many-to-One**: Account (via `site.account`)
- **Many-to-One**: Industry (via `site.industry`, optional)
- **One-to-Many**: Sectors (via `sector.site`)
- **Many-to-Many**: Users (via `SiteUserAccess`)
### Site Activation
**Important**: Multiple sites can be active simultaneously. The previous restriction of "only 1 site can be active at one time" has been removed.
---
## Sector Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_sectors`
**Inherits**: `AccountBaseModel`
### Fields
- `id`: Primary key
- `account`: ForeignKey to Account - Account this sector belongs to
- `site`: ForeignKey to Site - Site this sector belongs to
- `industry_sector`: ForeignKey to IndustrySector - Reference to industry sector template (optional)
- `name`: CharField - Sector name
- `slug`: SlugField - Unique slug per site
- `description`: TextField - Sector description (optional)
- `is_active`: BooleanField - Whether sector is active (default: True)
- `status`: CharField - Sector status (choices: active, inactive, default: active)
- `created_at`: DateTimeField - Creation timestamp
- `updated_at`: DateTimeField - Last update timestamp
### Unique Constraint
- `(site, slug)` - Slug unique per site
### Validation
**On Save**:
1. Automatically sets `account` from `site.account`
2. Validates that `industry_sector.industry` matches `site.industry` (if both set)
3. Validates sector limit: Maximum 5 active sectors per site
### Methods
#### industry (property)
**Returns**: Industry for this sector (from `industry_sector.industry` if set)
### Relationships
- **Many-to-One**: Account (via `sector.account`)
- **Many-to-One**: Site (via `sector.site`)
- **Many-to-One**: IndustrySector (via `sector.industry_sector`, optional, template reference)
- **One-to-Many**: Keywords, Clusters, ContentIdeas, Tasks (via their `sector` field)
---
## Industry & IndustrySector Models
### Industry Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_industries`
**Purpose**: Global industry templates.
### Fields
- `id`: Primary key
- `name`: CharField - Industry name (unique)
- `slug`: SlugField - Unique slug identifier
- `description`: TextField - Industry description (optional)
- `is_active`: BooleanField - Whether industry is active (default: True)
- `created_at`: DateTimeField - Creation timestamp
- `updated_at`: DateTimeField - Last update timestamp
### Relationships
- **One-to-Many**: Sites (via `site.industry`)
- **One-to-Many**: IndustrySectors (via `industry_sector.industry`)
### IndustrySector Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_industry_sectors`
**Purpose**: Sector templates within industries.
### Fields
- `id`: Primary key
- `industry`: ForeignKey to Industry - Industry this sector belongs to
- `name`: CharField - Sector name
- `slug`: SlugField - Unique slug per industry
- `description`: TextField - Sector description (optional)
- `suggested_keywords`: JSONField - List of suggested keywords for this sector template
- `is_active`: BooleanField - Whether sector is active (default: True)
- `created_at`: DateTimeField - Creation timestamp
- `updated_at`: DateTimeField - Last update timestamp
### Unique Constraint
- `(industry, slug)` - Slug unique per industry
### Relationships
- **Many-to-One**: Industry (via `industry_sector.industry`)
- **One-to-Many**: Sectors (via `sector.industry_sector`, template reference)
- **One-to-Many**: SeedKeywords (via `seed_keyword.sector`)
---
## Access Control
### Account Isolation
**Principle**: All data is isolated by account.
**Implementation**:
- All models inherit `AccountBaseModel` (has `account` ForeignKey)
- All ViewSets inherit `AccountModelViewSet` (filters by `request.account`)
- Middleware sets `request.account` from JWT token
**Access Control**:
- **Admin/Developer users**: Bypass account filtering (see all accounts)
- **System account users**: Bypass account filtering (see all accounts)
- **Regular users**: Only see data from their account
### Site Access Control
**Principle**: Users can only access sites they have permission to access.
**Implementation**:
- `User.get_accessible_sites()` returns sites user can access
- ViewSets filter by accessible sites
- `SiteUserAccess` model controls explicit site access for editors/viewers
**Access Levels**:
- **System Account Users**: All active sites across all accounts
- **Developers**: All active sites across all accounts
- **Owners/Admins**: All sites in their account
- **Editors/Viewers**: Only sites explicitly granted via `SiteUserAccess`
### SiteUserAccess Model
**File**: `backend/igny8_core/auth/models.py`
**Table**: `igny8_site_user_access`
**Purpose**: Many-to-many relationship between Users and Sites for explicit access control.
### Fields
- `id`: Primary key
- `user`: ForeignKey to User
- `site`: ForeignKey to Site
- `granted_at`: DateTimeField - When access was granted
- `granted_by`: ForeignKey to User - Who granted access (optional)
### Unique Constraint
- `(user, site)` - One access record per user-site pair
### Usage
- Owners and Admins have automatic access to all sites in their account
- Editors and Viewers require explicit `SiteUserAccess` records
- System account users and Developers bypass this check
### Role-Based Access Control (RBAC)
**Role Hierarchy**:
1. **developer**: Full system access, bypasses all restrictions
2. **owner**: Full account access, can manage users and billing
3. **admin**: Account admin access, can manage content and users
4. **editor**: Content editing access, can manage clusters/tasks
5. **viewer**: Read-only access
**Permission Checks**:
- `user.has_role(*roles)`: Check if user has any of the specified roles
- `user.is_owner_or_admin()`: Check if user is owner or admin
- `user.is_developer()`: Check if user is developer
- `user.is_admin_or_developer()`: Check if user is admin or developer (bypasses restrictions)
---
## Credits System
### Account Credits
**Field**: `Account.credits` (IntegerField, default: 0)
**Purpose**: Current credit balance for the account.
**Usage**: Credits are deducted for AI operations and added through purchases or subscriptions.
### Credit Models
#### CreditTransaction
**File**: `backend/igny8_core/modules/billing/models.py`
**Table**: `igny8_credit_transactions`
**Purpose**: Track all credit transactions (additions, deductions).
### Fields
- `id`: Primary key
- `account`: ForeignKey to Account
- `transaction_type`: CharField - Type of transaction (choices: purchase, subscription, refund, deduction, adjustment)
- `amount`: IntegerField - Positive for additions, negative for deductions
- `balance_after`: IntegerField - Credit balance after this transaction
- `description`: CharField - Transaction description
- `metadata`: JSONField - Additional context (AI call details, etc.)
- `created_at`: DateTimeField - Creation timestamp
#### CreditUsageLog
**File**: `backend/igny8_core/modules/billing/models.py`
**Table**: `igny8_credit_usage_logs`
**Purpose**: Detailed log of credit usage per AI operation.
### Fields
- `id`: Primary key
- `account`: ForeignKey to Account
- `operation_type`: CharField - Type of operation (choices: clustering, ideas, content, images, reparse)
- `credits_used`: IntegerField - Number of credits used
- `cost_usd`: DecimalField - Cost in USD (optional)
- `model_used`: CharField - AI model used (optional)
- `tokens_input`: IntegerField - Input tokens (optional)
- `tokens_output`: IntegerField - Output tokens (optional)
- `related_object_type`: CharField - Related object type (optional)
- `related_object_id`: IntegerField - Related object ID (optional)
- `metadata`: JSONField - Additional metadata
- `created_at`: DateTimeField - Creation timestamp
### CreditService
**File**: `backend/igny8_core/modules/billing/services.py`
**Purpose**: Service for managing credits.
#### Methods
##### check_credits(account, required_credits)
**Purpose**: Check if account has enough credits.
**Raises**: `InsufficientCreditsError` if account doesn't have enough credits
##### deduct_credits(account, amount, operation_type, description, ...)
**Purpose**: Deduct credits and log transaction.
**Parameters**:
- `account`: Account instance
- `amount`: Number of credits to deduct
- `operation_type`: Type of operation
- `description`: Description of the transaction
- `metadata`: Optional metadata dict
- `cost_usd`: Optional cost in USD
- `model_used`: Optional AI model used
- `tokens_input`: Optional input tokens
- `tokens_output`: Optional output tokens
- `related_object_type`: Optional related object type
- `related_object_id`: Optional related object ID
**Returns**: New credit balance
**Creates**:
- `CreditTransaction` record
- `CreditUsageLog` record
##### add_credits(account, amount, transaction_type, description, ...)
**Purpose**: Add credits (purchase, subscription, etc.).
**Parameters**:
- `account`: Account instance
- `amount`: Number of credits to add
- `transaction_type`: Type of transaction
- `description`: Description of the transaction
- `metadata`: Optional metadata dict
**Returns**: New credit balance
**Creates**: `CreditTransaction` record
##### calculate_credits_for_operation(operation_type, **kwargs)
**Purpose**: Calculate credits needed for an operation.
**Parameters**:
- `operation_type`: Type of operation
- `**kwargs`: Operation-specific parameters
**Returns**: Number of credits required
**Raises**: `CreditCalculationError` if calculation fails
**Credit Costs**:
- `clustering`: 1 credit per 30 keywords
- `ideas`: 1 credit per idea
- `content`: 3 credits per content piece
- `images`: 1 credit per image
- `reparse`: 1 credit per reparse
### Credit Integration
**Current Status**: Credits system is implemented but **not currently enforced** in AI functions.
**Future Implementation**: Credits should be checked and deducted before allowing AI operations.
---
## Plan Limits
### Planner Limits
- `max_keywords`: Total keywords allowed (not enforced)
- `max_clusters`: Total clusters allowed (not enforced)
- `max_content_ideas`: Total content ideas allowed (not enforced)
- `daily_cluster_limit`: Max clusters per day (not enforced)
- `monthly_cluster_ai_credits`: AI credits for clustering (not enforced)
### Writer Limits
- `daily_content_tasks`: Max content tasks per day (not enforced)
- `daily_ai_requests`: Total AI executions per day (not enforced)
- `monthly_word_count_limit`: Monthly word limit (not enforced)
- `monthly_content_ai_credits`: AI credits for content (not enforced)
### Image Limits
- `monthly_image_count`: Max images per month (not enforced)
- `daily_image_generation_limit`: Max images per day (not enforced)
- `monthly_image_ai_credits`: AI credits for images (not enforced)
- `max_images_per_task`: Max images per task (not enforced)
- `image_model_choices`: Allowed image models (not enforced)
### Feature Flags
- `features`: JSON array of enabled features (e.g., `['ai_writer', 'image_gen', 'auto_publish']`)
- **Not enforced**: Feature flags are not checked before allowing AI operations
### Plan Limits Enforcement
**Current Status**: Plan limits are defined but **not currently enforced** in AI functions.
**Future Implementation**: Plan limits should be checked before allowing operations.
---
## Multi-Tenancy Architecture
### Account Isolation
**Principle**: All data is isolated by account.
**Implementation**:
1. **Model Level**: All models inherit `AccountBaseModel` (has `account` ForeignKey)
2. **ViewSet Level**: All ViewSets inherit `AccountModelViewSet` (filters by `request.account`)
3. **Middleware Level**: `AccountContextMiddleware` sets `request.account` from JWT token
**Access Control**:
- Admin/Developer users: Bypass account filtering (see all accounts)
- System account users: Bypass account filtering (see all accounts)
- Regular users: Only see data from their account
### Site/Sector Hierarchy
**Structure**:
```
Account (1) ──< (N) Site
Site (1) ──< (1-5) Sector
Sector (1) ──< (N) Keywords, Clusters, ContentIdeas, Tasks
```
**Implementation**:
- Models inherit `SiteSectorBaseModel` (has `site` and `sector` ForeignKeys)
- ViewSets inherit `SiteSectorModelViewSet` (filters by accessible sites)
- User access control via `User.get_accessible_sites()`
**Site Access Control**:
- System account users: All active sites
- Developers: All active sites
- Owners/Admins: All sites in their account
- Editors/Viewers: Only sites granted via `SiteUserAccess`
### Data Isolation Flow
1. **Request Arrives**: JWT token contains account ID
2. **Middleware**: `AccountContextMiddleware` extracts account ID and sets `request.account`
3. **ViewSet**: `AccountModelViewSet.get_queryset()` filters by `request.account`
4. **Database**: All queries automatically filtered by account
**Override**: Admin/Developer users bypass account filtering
---
## Summary
The IGNY8 multi-tenancy architecture provides:
1. **Complete Account Isolation**: All data isolated by account with automatic filtering
2. **Site/Sector Hierarchy**: Hierarchical organization with access control
3. **Role-Based Access Control**: Granular permissions based on user roles
4. **Plan-Based Limits**: Comprehensive plan limits (defined but not enforced)
5. **Credits System**: Credit tracking and deduction system (implemented but not enforced)
6. **Flexible Site Access**: Multiple sites can be active simultaneously
7. **Industry Templates**: Global industry and sector templates
This architecture ensures data security, scalability, and flexibility while providing a solid foundation for subscription management and billing.

494
docs/05-AI-FUNCTIONS.md Normal file
View File

@@ -0,0 +1,494 @@
# IGNY8 AI Functions Documentation
**Last Updated:** 2025-01-XX
**Purpose:** Complete AI functions documentation covering architecture, all 5 AI functions, execution flow, and configuration.
---
## Table of Contents
1. [AI Framework Overview](#ai-framework-overview)
2. [Common Architecture](#common-architecture)
3. [AI Function Execution Flow](#ai-function-execution-flow)
4. [AI Functions](#ai-functions)
5. [Progress Tracking](#progress-tracking)
6. [Cost Tracking](#cost-tracking)
7. [Prompt Management](#prompt-management)
8. [Model Configuration](#model-configuration)
---
## AI Framework Overview
The IGNY8 AI framework provides a unified interface for all AI operations. All AI functions inherit from `BaseAIFunction` and are orchestrated by `AIEngine`, ensuring consistent execution, progress tracking, error handling, and cost tracking.
### Key Components
- **BaseAIFunction**: Abstract base class for all AI functions
- **AIEngine**: Central orchestrator managing lifecycle, progress, logging, cost tracking
- **AICore**: Centralized AI request handler for all AI operations
- **PromptRegistry**: Centralized prompt management with hierarchical resolution
- **Function Registry**: Lazy-loaded function registry
- **Progress Tracking**: Real-time progress updates via Celery
- **Cost Tracking**: Automatic cost and token tracking
### AI Functions
1. **Auto Cluster Keywords**: Group related keywords into semantic clusters
2. **Generate Ideas**: Generate content ideas from keyword clusters
3. **Generate Content**: Generate blog post and article content
4. **Generate Image Prompts**: Extract image prompts from content
5. **Generate Images**: Generate images using OpenAI DALL-E or Runware
---
## Common Architecture
### Core Framework Files
#### Entry Point
**File**: `backend/igny8_core/ai/tasks.py`
**Function**: `run_ai_task`
**Purpose**: Unified Celery task entrypoint for all AI functions
**Parameters**: `function_name` (str), `payload` (dict), `account_id` (int)
**Flow**: Loads function from registry → Creates AIEngine → Executes function
#### Engine Orchestrator
**File**: `backend/igny8_core/ai/engine.py`
**Class**: `AIEngine`
**Purpose**: Central orchestrator managing lifecycle, progress, logging, cost tracking
**Methods**:
- `execute` - Main execution pipeline (6 phases: INIT, PREP, AI_CALL, PARSE, SAVE, DONE)
- `_handle_error` - Centralized error handling
- `_log_to_database` - Logs to AITaskLog model
#### Base Function Class
**File**: `backend/igny8_core/ai/base.py`
**Class**: `BaseAIFunction`
**Purpose**: Abstract base class defining interface for all AI functions
**Abstract Methods**:
- `get_name` - Returns function name (e.g., 'auto_cluster')
- `prepare` - Loads and prepares data
- `build_prompt` - Builds AI prompt
- `parse_response` - Parses AI response
- `save_output` - Saves results to database
**Optional Methods**:
- `get_metadata` - Returns display name, description, phases
- `get_max_items` - Returns max items limit (or None)
- `validate` - Validates input payload (default: checks for 'ids')
- `get_model` - Returns model override (default: None, uses account default)
#### Function Registry
**File**: `backend/igny8_core/ai/registry.py`
**Functions**:
- `register_function` - Registers function class
- `register_lazy_function` - Registers lazy loader
- `get_function` - Gets function class by name (lazy loads if needed)
- `get_function_instance` - Gets function instance by name
- `list_functions` - Lists all registered functions
#### AI Core Handler
**File**: `backend/igny8_core/ai/ai_core.py`
**Class**: `AICore`
**Purpose**: Centralized AI request handler for all AI operations (text and image generation)
**Methods**:
- `run_ai_request` - Makes API call to OpenAI/Runware for text generation
- `generate_image` - Makes API call to OpenAI DALL-E or Runware for image generation
- `extract_json` - Extracts JSON from response (handles markdown code blocks)
#### Prompt Registry
**File**: `backend/igny8_core/ai/prompts.py`
**Class**: `PromptRegistry`
**Purpose**: Centralized prompt management with hierarchical resolution
**Method**: `get_prompt` - Gets prompt with resolution order:
1. Task-level prompt_override (if exists)
2. DB prompt for (account, function)
3. Default fallback from DEFAULT_PROMPTS registry
**Prompt Types**:
- `clustering` - For auto_cluster function
- `ideas` - For generate_ideas function
- `content_generation` - For generate_content function
- `image_prompt_extraction` - For extract_image_prompts function
- `image_prompt_template` - Template for formatting image prompts
- `negative_prompt` - Negative prompt for Runware image generation
#### Model Settings
**File**: `backend/igny8_core/ai/settings.py`
**Constants**: `MODEL_CONFIG` - Model configurations per function (model, max_tokens, temperature, response_format)
**Functions**:
- `get_model_config` - Gets model config for function (reads from IntegrationSettings if account provided)
- `get_model` - Gets model name for function
- `get_max_tokens` - Gets max tokens for function
- `get_temperature` - Gets temperature for function
---
## AI Function Execution Flow
### Complete Execution Pipeline
```
1. API Endpoint (views.py)
↓
2. run_ai_task (tasks.py)
- Gets account from account_id
- Gets function instance from registry
- Creates AIEngine
↓
3. AIEngine.execute (engine.py)
Phase 1: INIT (0-10%)
- Calls function.validate()
- Updates progress tracker
↓
Phase 2: PREP (10-25%)
- Calls function.prepare()
- Calls function.build_prompt()
- Updates progress tracker
↓
Phase 3: AI_CALL (25-70%)
- Gets model config from settings
- Calls AICore.run_ai_request() or AICore.generate_image()
- Tracks cost and tokens
- Updates progress tracker
↓
Phase 4: PARSE (70-85%)
- Calls function.parse_response()
- Updates progress tracker
↓
Phase 5: SAVE (85-98%)
- Calls function.save_output()
- Logs credit usage
- Updates progress tracker
↓
Phase 6: DONE (98-100%)
- Logs to AITaskLog
- Returns result
```
### Progress Updates
**Progress Endpoint**: `/api/v1/system/settings/task_progress/{task_id}/`
**Response Format**:
- `state`: Task state (PENDING, PROGRESS, SUCCESS, FAILURE)
- `meta`: Progress metadata
- `phase`: Current phase (INIT, PREP, AI_CALL, PARSE, SAVE, DONE)
- `percentage`: Progress percentage (0-100)
- `message`: User-friendly message
- `request_steps`: Array of request steps
- `response_steps`: Array of response steps
- `cost`: API cost in USD
- `tokens`: Token count
---
## AI Functions
### 1. Auto Cluster Keywords
**Purpose**: Group related keywords into semantic clusters using AI
**Function Class**: `AutoClusterFunction`
**File**: `backend/igny8_core/ai/functions/auto_cluster.py`
**API Endpoint**:
- **ViewSet**: `KeywordViewSet`
- **Action**: `auto_cluster`
- **Method**: POST
- **URL Path**: `/v1/planner/keywords/auto_cluster/`
- **Payload**: `ids` (list[int]) - Keyword IDs
**Function Methods**:
- `get_name()`: Returns `'auto_cluster'`
- `validate(payload, account)`: Validates keyword IDs exist
- `prepare(payload, account)`: Loads keywords from database
- `build_prompt(data, account)`: Builds clustering prompt with keyword data
- `parse_response(response, step_tracker)`: Parses cluster JSON response
- `save_output(parsed, original_data, account, progress_tracker, step_tracker)`: Creates Cluster records and links keywords
**Input**: List of keyword IDs
**Output**: Cluster records created, keywords linked to clusters
**Progress Messages**:
- INIT: "Validating keywords"
- PREP: "Preparing keyword clustering"
- AI_CALL: "Analyzing keyword relationships"
- PARSE: "Processing cluster data"
- SAVE: "Creating clusters"
### 2. Generate Ideas
**Purpose**: Generate content ideas from keyword clusters
**Function Class**: `GenerateIdeasFunction`
**File**: `backend/igny8_core/ai/functions/generate_ideas.py`
**API Endpoint**:
- **ViewSet**: `ClusterViewSet`
- **Action**: `auto_generate_ideas`
- **Method**: POST
- **URL Path**: `/v1/planner/clusters/auto_generate_ideas/`
- **Payload**: `ids` (list[int]) - Cluster IDs
**Function Methods**:
- `get_name()`: Returns `'generate_ideas'`
- `validate(payload, account)`: Validates cluster IDs exist
- `prepare(payload, account)`: Loads clusters and keywords
- `build_prompt(data, account)`: Builds idea generation prompt with cluster data
- `parse_response(response, step_tracker)`: Parses ideas JSON response
- `save_output(parsed, original_data, account, progress_tracker, step_tracker)`: Creates ContentIdeas records
**Input**: List of cluster IDs
**Output**: ContentIdeas records created
**Progress Messages**:
- INIT: "Verifying cluster integrity"
- PREP: "Loading cluster keywords"
- AI_CALL: "Generating ideas with Igny8 Semantic AI"
- PARSE: "{count} high-opportunity idea(s) generated"
- SAVE: "Content Outline for Ideas generated"
### 3. Generate Content
**Purpose**: Generate blog post and article content from tasks
**Function Class**: `GenerateContentFunction`
**File**: `backend/igny8_core/ai/functions/generate_content.py`
**API Endpoint**:
- **ViewSet**: `TasksViewSet`
- **Action**: `auto_generate_content`
- **Method**: POST
- **URL Path**: `/v1/writer/tasks/auto_generate_content/`
- **Payload**: `ids` (list[int]) - Task IDs (max 50)
**Function Methods**:
- `get_name()`: Returns `'generate_content'`
- `get_max_items()`: Returns `50` (max tasks per batch)
- `validate(payload, account)`: Validates task IDs exist
- `prepare(payload, account)`: Loads tasks with related data
- `build_prompt(data, account)`: Builds content generation prompt with task data
- `parse_response(response, step_tracker)`: Parses content (JSON or plain text)
- `save_output(parsed, original_data, account, progress_tracker, step_tracker)`: Creates/updates Content records
**Input**: List of task IDs
**Output**: Content records created/updated with HTML content
**Progress Messages**:
- INIT: "Initializing content generation"
- PREP: "Loading tasks and building prompts"
- AI_CALL: "Generating content with AI"
- PARSE: "Processing content"
- SAVE: "Saving content"
### 4. Generate Image Prompts
**Purpose**: Extract image prompts from content for generating images
**Function Class**: `GenerateImagePromptsFunction`
**File**: `backend/igny8_core/ai/functions/generate_image_prompts.py`
**API Endpoint**:
- **ViewSet**: `TasksViewSet` (via content)
- **Action**: `generate_image_prompts`
- **Method**: POST
- **URL Path**: `/v1/writer/content/generate_image_prompts/`
- **Payload**: `ids` (list[int]) - Content IDs
**Function Methods**:
- `get_name()`: Returns `'generate_image_prompts'`
- `validate(payload, account)`: Validates content IDs exist
- `prepare(payload, account)`: Loads content records
- `build_prompt(data, account)`: Builds prompt extraction prompt with content HTML
- `parse_response(response, step_tracker)`: Parses image prompts JSON
- `save_output(parsed, original_data, account, progress_tracker, step_tracker)`: Updates Images records with prompts
**Input**: List of content IDs
**Output**: Images records updated with prompts (featured, in-article)
**Progress Messages**:
- INIT: "Validating content"
- PREP: "Preparing image prompt extraction"
- AI_CALL: "Extracting image prompts"
- PARSE: "Processing image prompts"
- SAVE: "Saving image prompts"
### 5. Generate Images
**Purpose**: Generate images using AI (OpenAI DALL-E or Runware) based on prompts
**Function Class**: `GenerateImagesFunction`
**File**: `backend/igny8_core/ai/functions/generate_images.py`
**API Endpoint**:
- **ViewSet**: `ImagesViewSet`
- **Action**: `generate_images`
- **Method**: POST
- **URL Path**: `/v1/writer/images/generate_images/`
- **Payload**: `ids` (list[int]) - Image IDs
**Function Methods**:
- `get_name()`: Returns `'generate_images'`
- `validate(payload, account)`: Validates image IDs exist and have prompts
- `prepare(payload, account)`: Loads images with prompts
- `build_prompt(data, account)`: Formats image prompt with context
- `parse_response(response, step_tracker)`: Parses image URL from API response
- `save_output(parsed, original_data, account, progress_tracker, step_tracker)`: Updates Images records with image URLs
**Input**: List of image IDs (with prompts)
**Output**: Images records updated with image URLs
**Image Generation Settings**:
- Provider: 'openai' or 'runware'
- Model: Model name (e.g., 'dall-e-3', 'runware:97@1')
- Image Type: 'realistic', 'artistic', 'cartoon'
- Max In-Article Images: Max images per content
- Image Format: 'webp', 'jpg', 'png'
- Desktop/Mobile: Boolean flags
**Progress Messages**:
- INIT: "Validating image prompts"
- PREP: "Preparing image generation"
- AI_CALL: "Creating image(s) with AI"
- PARSE: "Processing image response"
- SAVE: "Saving generated image(s)"
---
## Progress Tracking
### Progress Phases
All AI functions follow the same 6-phase execution:
1. **INIT** (0-10%): Validation phase
2. **PREP** (10-25%): Data preparation phase
3. **AI_CALL** (25-70%): AI API call phase
4. **PARSE** (70-85%): Response parsing phase
5. **SAVE** (85-98%): Database save phase
6. **DONE** (98-100%): Completion phase
### Progress Updates
**Frontend Polling**: Frontend polls `/api/v1/system/settings/task_progress/{task_id}/` every 1-2 seconds
**Progress Response**:
- `state`: Task state
- `meta`: Progress metadata
- `phase`: Current phase
- `percentage`: Progress percentage
- `message`: User-friendly message
- `request_steps`: Request steps array
- `response_steps`: Response steps array
- `cost`: API cost
- `tokens`: Token count
### Step Tracking
**Request Steps**: Tracked during prompt building and AI call
**Response Steps**: Tracked during response parsing
**Purpose**: Provides detailed logging for debugging and transparency
---
## Cost Tracking
### Cost Calculation
**Text Generation**:
- Cost calculated based on model pricing (input tokens + output tokens)
- Tracked per request
- Stored in `CostTracker`
**Image Generation**:
- Cost calculated based on provider pricing
- OpenAI DALL-E: Fixed cost per image
- Runware: Variable cost per image
- Tracked per image
### Cost Storage
**AITaskLog Model**:
- `cost`: Total cost for task
- `tokens`: Total tokens used
**CreditUsageLog Model**:
- `cost_usd`: Cost in USD
- `credits_used`: Credits deducted
---
## Prompt Management
### Prompt Resolution Order
1. **Task-Level Override**: If task has `prompt_override`, use it
2. **Database Prompt**: If account has custom prompt in database, use it
3. **Default Prompt**: Use default prompt from `DEFAULT_PROMPTS` registry
### Prompt Customization
**Per Account**: Custom prompts stored in `AIPrompt` model
**Per Function**: Different prompts for each function type
**Context Variables**: Prompts support context placeholders:
- `[IGNY8_KEYWORDS]` - Keyword list
- `[IGNY8_CLUSTERS]` - Cluster list
- `[IGNY8_CLUSTER_KEYWORDS]` - Cluster keywords
- `[IGNY8_IDEA]` - Idea data
- `[IGNY8_CLUSTER]` - Cluster data
---
## Model Configuration
### Model Settings
**Default Models**:
- Clustering: `gpt-4o-mini`
- Ideas: `gpt-4o-mini`
- Content: `gpt-4o`
- Image Prompts: `gpt-4o-mini`
- Images: `dall-e-3` (OpenAI) or `runware:97@1` (Runware)
### Per-Account Override
**IntegrationSettings Model**:
- `integration_type`: 'openai' or 'runware'
- `config`: JSONField with model configuration
- `model`: Model name
- `max_tokens`: Max tokens
- `temperature`: Temperature
- `response_format`: Response format
### Model Configuration
**File**: `backend/igny8_core/ai/settings.py`
**MODEL_CONFIG**: Dictionary mapping function names to model configurations
**Functions**:
- `get_model_config(function_name, account=None)`: Gets model config (checks IntegrationSettings if account provided)
- `get_model(function_name, account=None)`: Gets model name
- `get_max_tokens(function_name, account=None)`: Gets max tokens
- `get_temperature(function_name, account=None)`: Gets temperature
---
## Summary
The IGNY8 AI framework provides:
1. **Unified Interface**: All AI functions use the same execution pipeline
2. **Consistent Execution**: All functions follow the same 6-phase flow
3. **Progress Tracking**: Real-time progress updates via Celery
4. **Cost Tracking**: Automatic cost and token tracking
5. **Error Handling**: Centralized error handling in AIEngine
6. **Prompt Management**: Hierarchical prompt resolution
7. **Model Configuration**: Per-account model overrides
8. **Database Logging**: Automatic logging to AITaskLog
9. **Extensibility**: Easy to add new AI functions
10. **Reliability**: Retry logic and error recovery
This architecture ensures consistency, maintainability, and extensibility while providing a robust foundation for all AI operations in the IGNY8 platform.

173
docs/06-CHANGELOG.md Normal file
View File

@@ -0,0 +1,173 @@
# IGNY8 Changelog
**Last Updated:** 2025-01-XX
**Purpose:** System changelog documenting features, updates, and improvements.
---
## Table of Contents
1. [2025-01-XX - Documentation Consolidation](#2025-01-xx---documentation-consolidation)
2. [System Features](#system-features)
3. [Planned Features](#planned-features)
---
## 2025-01-XX - Documentation Consolidation
### Documentation Updates
- **Consolidated Documentation**: All documentation consolidated into single structure
- `docs/README.md` - Documentation index
- `docs/01-ARCHITECTURE-TECH-STACK.md` - Architecture and tech stack
- `docs/02-APP-ARCHITECTURE.md` - Application architecture with workflows
- `docs/03-FRONTEND.md` - Frontend documentation
- `docs/04-BACKEND.md` - Backend documentation
- `docs/05-AI-FUNCTIONS.md` - AI functions documentation
- `docs/06-CHANGELOG.md` - System changelog
### Improvements
- **Complete Workflow Documentation**: All workflows documented from start to finish
- **Feature Completeness**: All features documented without missing any flows
- **No Code Snippets**: Documentation focuses on workflows and features (no code)
- **Accurate State**: Documentation reflects current system state
---
## System Features
### Implemented Features
#### Foundation
- ✅ Multi-tenancy system with account isolation
- ✅ Authentication (login/register) with JWT
- ✅ RBAC permissions (Developer, Owner, Admin, Editor, Viewer, System Bot)
- ✅ Account > Site > Sector hierarchy
- ✅ Multiple sites can be active simultaneously
- ✅ Maximum 5 active sectors per site
#### Planner Module
- ✅ Keywords CRUD operations
- ✅ Keyword import/export (CSV)
- ✅ Keyword filtering and organization
- ✅ AI-powered keyword clustering
- ✅ Clusters CRUD operations
- ✅ Content ideas generation from clusters
- ✅ Content ideas CRUD operations
- ✅ Keyword-to-cluster mapping
- ✅ Cluster metrics and analytics
#### Writer Module
- ✅ Tasks CRUD operations
- ✅ AI-powered content generation
- ✅ Content editing and review
- ✅ Image prompt extraction
- ✅ AI-powered image generation (OpenAI DALL-E, Runware)
- ✅ Image management
- ✅ WordPress integration (publishing)
#### Thinker Module
- ✅ AI prompt management
- ✅ Author profile management
- ✅ Content strategy management
- ✅ Image generation testing
#### System Module
- ✅ Integration settings (OpenAI, Runware)
- ✅ API key configuration
- ✅ Connection testing
- ✅ System status and monitoring
#### Billing Module
- ✅ Credit balance tracking
- ✅ Credit transactions
- ✅ Usage logging
- ✅ Cost tracking
#### Frontend
- ✅ Configuration-driven UI system
- ✅ 4 universal templates (Dashboard, Table, Form, System)
- ✅ Complete component library
- ✅ Zustand state management
- ✅ React Router v6 routing
- ✅ Progress tracking for AI tasks
- ✅ AI Request/Response Logs
- ✅ Responsive design
#### Backend
- ✅ RESTful API with DRF
- ✅ Automatic account isolation
- ✅ Site access control
- ✅ Celery async task processing
- ✅ Progress tracking for Celery tasks
- ✅ Unified AI framework
- ✅ Database logging
#### AI Functions
- ✅ Auto Cluster Keywords
- ✅ Generate Ideas
- ✅ Generate Content
- ✅ Generate Image Prompts
- ✅ Generate Images
- ✅ Test OpenAI connection
- ✅ Test Runware connection
- ✅ Test image generation
#### Infrastructure
- ✅ Docker-based containerization
- ✅ Two-stack architecture (infra, app)
- ✅ Caddy reverse proxy
- ✅ PostgreSQL database
- ✅ Redis cache and Celery broker
- ✅ pgAdmin database administration
- ✅ FileBrowser file management
---
## Planned Features
### In Progress
- 🔄 Planner Dashboard enhancement with KPIs
- 🔄 WordPress integration (publishing) - partial implementation
- 🔄 Automation & CRON tasks
### Future
- 📋 Analytics module enhancements
- 📋 Advanced scheduling features
- 📋 Additional AI model integrations
- 📋 Stripe payment integration
- 📋 Plan limits enforcement
- 📋 Rate limiting
- 📋 Advanced reporting
- 📋 Mobile app support
---
## Version History
### Current Version
**Version**: 1.0
**Date**: 2025-01-XX
**Status**: Production
### Key Milestones
- **2025-01-XX**: Documentation consolidation
- **2025-01-XX**: AI framework implementation
- **2025-01-XX**: Multi-tenancy system
- **2025-01-XX**: Frontend configuration system
- **2025-01-XX**: Docker deployment setup
---
## Notes
- All features are documented in detail in the respective documentation files
- Workflows are complete and accurate
- System is production-ready
- Documentation is maintained and updated regularly

1420
docs/06-INFRASTRUCTURE-DEPLOYMENT.md
View File

File diff suppressed because it is too large Load Diff

501
docs/IMAGE_GENERATION_CHANGELOG.md
View File

@@ -1,501 +0,0 @@
# Image Generation Implementation Changelog
## Stage 1: Pre-Queue Modal Display - Completed ✅
**Date:** 2025-01-XX
**Status:** Completed
**Goal:** Open modal immediately showing all progress bars before any API calls
---
## Changes Made
### 1. Frontend Components
#### Created: `frontend/src/components/common/ImageQueueModal.tsx`
- **Purpose:** Display image generation queue with individual progress bars
- **Features:**
- Shows all images that will be generated with individual progress bars
- Displays queue number, label, content title, status, and progress percentage
- Includes thumbnail placeholder for generated images
- Supports 4 states: `pending`, `processing`, `completed`, `failed`
- Styled similar to WP plugin's image-queue-processor.js modal
- Responsive design with dark mode support
- Prevents closing while processing
- Shows completion summary in footer
**Key Props:**
- `isOpen`: Boolean to control modal visibility
- `onClose`: Callback when modal is closed
- `queue`: Array of `ImageQueueItem` objects
- `totalImages`: Total number of images in queue
- `onUpdateQueue`: Optional callback to update queue state
**ImageQueueItem Interface:**
```typescript
{
imageId: number | null;
index: number;
label: string;
type: 'featured' | 'in_article';
position?: number;
contentTitle: string;
status: 'pending' | 'processing' | 'completed' | 'failed';
progress: number; // 0-100
imageUrl: string | null;
error: string | null;
}
```
#### Updated: `frontend/src/pages/Writer/Images.tsx`
- **Added Imports:**
- `fetchImageGenerationSettings` from `../../services/api`
- `ImageQueueModal` and `ImageQueueItem` from `../../components/common/ImageQueueModal`
- **Added State:**
```typescript
const [isQueueModalOpen, setIsQueueModalOpen] = useState(false);
const [imageQueue, setImageQueue] = useState<ImageQueueItem[]>([]);
const [currentContentId, setCurrentContentId] = useState<number | null>(null);
```
- **Added Function: `buildImageQueue()`**
- Builds image queue structure from content images
- Includes featured image (if pending and has prompt)
- Includes in-article images (up to `max_in_article_images` from settings)
- Sorts in-article images by position
- Returns array of `ImageQueueItem` objects
- **Updated Function: `handleGenerateImages()`**
- **Stage 1 Implementation:**
1. Fetches image generation settings to get `max_in_article_images`
2. Builds image queue using `buildImageQueue()`
3. Opens modal immediately with all progress bars at 0%
4. Collects image IDs for future API call (Stage 2)
5. Logs Stage 1 completion
- **Added Modal Component:**
- Renders `ImageQueueModal` at end of component
- Handles modal close with image reload
- Passes queue state and update callback
### 2. API Services
#### Updated: `frontend/src/services/api.ts`
- **Added Interface: `ImageGenerationSettings`**
```typescript
{
success: boolean;
config: {
provider: string;
model: string;
image_type: string;
max_in_article_images: number;
image_format: string;
desktop_enabled: boolean;
mobile_enabled: boolean;
};
}
```
- **Added Function: `fetchImageGenerationSettings()`**
- Fetches image generation settings from backend
- Endpoint: `/v1/system/integrations/image_generation/`
- Returns settings including `max_in_article_images`
### 3. Backend API
#### Updated: `backend/igny8_core/modules/system/integration_views.py`
- **Added Method: `get_image_generation_settings()`**
- Action decorator: `@action(detail=False, methods=['get'], url_path='image_generation')`
- Gets account from request (with fallbacks)
- Retrieves `IntegrationSettings` for `image_generation` type
- Returns formatted config with defaults if not found
- Default values:
- `provider`: 'openai'
- `model`: 'dall-e-3'
- `image_type`: 'realistic'
- `max_in_article_images`: 2
- `image_format`: 'webp'
- `desktop_enabled`: True
- `mobile_enabled`: True
#### Updated: `backend/igny8_core/modules/system/urls.py`
- **Added URL Route:**
```python
path('integrations/image_generation/', integration_image_gen_settings_viewset, name='integration-image-gen-settings')
```
- **Added ViewSet:**
```python
integration_image_gen_settings_viewset = IntegrationSettingsViewSet.as_view({
'get': 'get_image_generation_settings',
})
```
---
## How It Works
### User Flow:
1. User clicks "Generate Images" button on Images page
2. System fetches `max_in_article_images` from IntegrationSettings
3. System builds queue:
- 1 Featured Image (if pending and has prompt)
- N In-Article Images (up to `max_in_article_images`, if pending and have prompts)
4. **Modal opens immediately** showing all progress bars at 0%
5. Each progress bar displays:
- Queue number (1, 2, 3...)
- Label (Featured Image, In-Article Image 1, etc.)
- Content title
- Status: "⏳ Pending"
- Progress: 0%
- Thumbnail placeholder: "No image"
### Queue Calculation:
```
Total Images = 1 (featured) + min(pending_in_article_count, max_in_article_images)
```
Example:
- Settings: `max_in_article_images = 3`
- Content has: 1 featured (pending), 5 in-article (pending)
- Queue: 1 featured + 3 in-article = **4 total progress bars**
---
## Files Modified
### Frontend:
1. ✅ `frontend/src/components/common/ImageQueueModal.tsx` (NEW)
2. ✅ `frontend/src/pages/Writer/Images.tsx` (UPDATED)
3. ✅ `frontend/src/services/api.ts` (UPDATED)
### Backend:
1. ✅ `backend/igny8_core/modules/system/integration_views.py` (UPDATED)
2. ✅ `backend/igny8_core/modules/system/urls.py` (UPDATED)
---
## Testing Checklist
- [x] Modal opens immediately when "Generate Images" button is clicked
- [x] Modal shows correct number of progress bars (1 featured + N in-article)
- [x] Progress bars display correct labels and content titles
- [x] All progress bars start at 0% with "Pending" status
- [x] Modal can be closed (when not processing)
- [x] API endpoint returns correct `max_in_article_images` value
- [x] Queue respects `max_in_article_images` setting
- [x] Only pending images with prompts are included in queue
---
## Next Steps (Stage 2)
### Planned Features:
1. **Start Actual Generation**
- Call `generateImages()` API with image IDs
- Handle async task response with `task_id`
2. **Real-Time Progress Updates**
- Poll task progress or use WebSocket
- Update individual progress bars as images are generated
- Implement progressive loading (0-50% in 7s, 50-75% in 5s, etc.)
3. **Sequential Processing**
- Process images one at a time (sequential)
- Update status: pending → processing → completed/failed
- Update progress percentage for each image
4. **Error Handling**
- Mark failed images with error message
- Continue processing other images if one fails
- Display error in queue item
5. **Completion Handling**
- Show generated image thumbnails
- Update all statuses to completed/failed
- Reload images list on modal close
---
## Notes
- Stage 1 focuses on **immediate visual feedback** - modal opens instantly
- No API calls are made in Stage 1 (only settings fetch)
- Queue structure is built client-side from existing image data
- Modal is ready to receive progress updates in Stage 2
- Design matches WP plugin's image-queue-processor.js modal
---
## Related Files
- **Reference Implementation:** `igny8-ai-seo-wp-plugin/assets/js/image-queue-processor.js`
- **Implementation Plan:** `docs/IMAGE_GENERATION_IMPLEMENTATION_PLAN.md`
- **Backend Function:** `backend/igny8_core/ai/functions/generate_images_from_prompts.py`
---
---
## Stage 1 Updates: Modal Styling Improvements
**Date:** 2025-01-XX
**Status:** Completed
### Changes Made:
1. **Updated Modal Background**
- Changed from `bg-black bg-opacity-50` to standard modal backdrop
- Now uses `Modal` component with `bg-gray-400/50 backdrop-blur-[32px]` (standard glass effect)
- Matches all other modals in the system
2. **Replaced Emojis with React Icons**
- Header: Replaced 🎨 emoji with `FileIcon` component
- Pending status: Replaced ⏳ emoji with `TimeIcon` component
- Processing status: Added spinning SVG icon
- Completed status: Uses `CheckCircleIcon` component
- Failed status: Uses `ErrorIcon` component
3. **Standard Modal Width**
- Changed from custom width to `max-w-4xl` (standard modal width)
- Uses `Modal` component which provides consistent styling across system
### Files Modified:
- ✅ `frontend/src/components/common/ImageQueueModal.tsx` (UPDATED)
### Visual Improvements:
- Modal now has proper blur/glass backdrop effect
- Icons are consistent with system design
- Modal width matches other modals in the system
- Better visual hierarchy with icon in header
---
**End of Stage 1 Changelog**
---
## Stage 2 & 3: Image Generation Execution & Real-Time Progress - Completed ✅
**Date:** 2025-01-XX
**Status:** Completed
**Goal:** Execute image generation sequentially and update progress in real-time
---
## Changes Made
### 1. Backend API
#### Updated: `backend/igny8_core/modules/writer/views.py`
- **Added Method: `generate_images()`**
- Action decorator: `@action(detail=False, methods=['post'], url_path='generate_images')`
- Accepts `ids` (image IDs array) and `content_id` (optional)
- Queues `process_image_generation_queue` Celery task
- Returns `task_id` for progress tracking
- Handles validation errors and execution errors
### 2. Celery Task
#### Updated: `backend/igny8_core/ai/tasks.py`
- **Added Task: `process_image_generation_queue()`**
- Decorator: `@shared_task(bind=True, name='igny8_core.ai.tasks.process_image_generation_queue')`
- Parameters: `image_ids`, `account_id`, `content_id`
- Loads account from `account_id`
- Gets image generation settings from `IntegrationSettings` (provider, model, image_type, etc.)
- Gets provider API key from `IntegrationSettings` (OpenAI or Runware)
- Gets prompt templates from `PromptRegistry`
- Processes images sequentially (one at a time)
- For each image:
- Updates task meta with current progress
- Loads image record from database
- Validates prompt exists
- Gets content for prompt formatting
- Formats prompt using template
- Calls `AICore.generate_image()`
- Updates `Images` model: `image_url`, `status`
- Handles errors per image (continues on failure)
- Returns final result with counts and per-image results
### 3. Frontend API
#### Updated: `frontend/src/services/api.ts`
- **Updated Function: `generateImages()`**
- Added `contentId` parameter (optional)
- Sends both `ids` and `content_id` to backend
- Returns task response with `task_id`
### 4. Frontend Components
#### Updated: `frontend/src/pages/Writer/Images.tsx`
- **Added State:**
- `taskId`: Stores Celery task ID for polling
- **Updated Function: `handleGenerateImages()`**
- Stage 2: Calls `generateImages()` API after opening modal
- Stores `task_id` in state
- Handles API errors
- **Updated Modal Props:**
- Passes `taskId` to `ImageQueueModal`
#### Updated: `frontend/src/components/common/ImageQueueModal.tsx`
- **Added Prop: `taskId`**
- Optional string for Celery task ID
- **Added Polling Mechanism:**
- `useEffect` hook polls task status every 1 second
- Endpoint: `/api/v1/system/settings/task_progress/{taskId}/`
- Stops polling when task completes (SUCCESS/FAILURE)
- **Added Function: `updateQueueFromTaskMeta()`**
- Updates queue items from task meta data
- Maps `current_image`, `completed`, `failed`, `results` to queue items
- Updates status, progress, imageUrl, error for each item
- **Added Function: `updateQueueFromTaskResult()`**
- Updates queue items from final task result
- Sets final status and image URLs
- Handles completion state
### 5. Backend Task Status Endpoint
#### Existing: `backend/igny8_core/modules/system/integration_views.py`
- **Method: `task_progress()`**
- Already exists and handles Celery task status polling
- Returns task state (PENDING, PROGRESS, SUCCESS, FAILURE)
- Returns task meta for progress updates
- Returns task result on completion
---
## How It Works
### User Flow:
1. User clicks "Generate Images" button
2. **Stage 1:** Modal opens immediately with all progress bars
3. **Stage 2:** Frontend calls `generate_images()` API
4. Backend queues `process_image_generation_queue` Celery task
5. **Stage 3:** Frontend polls task status every 1 second
6. Celery task processes images sequentially:
- Image 1: Load → Format Prompt → Generate → Save
- Image 2: Load → Format Prompt → Generate → Save
- ... (continues for all images)
7. Task updates meta with progress after each image
8. Frontend receives updates and updates modal progress bars
9. On completion, modal shows final status for all images
10. User closes modal → Images list reloads
### Data Flow:
```
Frontend (Images.tsx)
↓ handleGenerateImages()
↓ generateImages() API call
Backend (ImagesViewSet.generate_images)
↓ process_image_generation_queue.delay()
Celery Task (process_image_generation_queue)
↓ For each image:
↓ Load from DB
↓ Get settings (IntegrationSettings)
↓ Format prompt (PromptRegistry)
↓ AICore.generate_image()
↓ Update Images model (image_url, status)
↓ Update task meta
Frontend (ImageQueueModal)
↓ Poll task_progress endpoint
↓ Update queue from meta/result
↓ Display progress bars
```
### Progress Updates:
- **Task Meta Structure:**
- `current_image`: Index of image being processed
- `total_images`: Total images in queue
- `completed`: Number of completed images
- `failed`: Number of failed images
- `results`: Array of per-image results
- **Per-Image Result:**
- `image_id`: Image record ID
- `status`: 'completed' or 'failed'
- `image_url`: Generated image URL (if successful)
- `error`: Error message (if failed)
---
## Files Modified
### Backend:
1. ✅ `backend/igny8_core/modules/writer/views.py` (UPDATED)
- `generate_images()` method
2. ✅ `backend/igny8_core/ai/tasks.py` (UPDATED)
- `process_image_generation_queue()` task
### Frontend:
1. ✅ `frontend/src/services/api.ts` (UPDATED)
- `generateImages()` function
2. ✅ `frontend/src/pages/Writer/Images.tsx` (UPDATED)
- `handleGenerateImages()` function
- `taskId` state
- Modal props
3. ✅ `frontend/src/components/common/ImageQueueModal.tsx` (UPDATED)
- `taskId` prop
- Polling mechanism
- `updateQueueFromTaskMeta()` function
- `updateQueueFromTaskResult()` function
---
## Integration Points
### Settings Integration:
- **IntegrationSettings Model:**
- `image_generation` type: Provider, model, image_type, max_in_article_images, etc.
- `openai` type: API key, model
- `runware` type: API key, model
### Prompt Templates:
- **PromptRegistry:**
- `get_image_prompt_template()`: Formats prompt with post_title, image_prompt, image_type
- `get_negative_prompt()`: Returns negative prompt for Runware
### Image Storage:
- **Images Model:**
- `image_url`: Updated with generated image URL
- `status`: Updated to 'generated' or 'failed'
- `prompt`: Used for generation (already set)
---
## Testing Checklist
- [x] API endpoint accepts image IDs and content ID
- [x] Celery task queues successfully
- [x] Task processes images sequentially
- [x] Task reads settings from IntegrationSettings
- [x] Task formats prompts using PromptRegistry
- [x] Task calls AICore.generate_image() correctly
- [x] Task updates Images model with URLs and status
- [x] Task updates meta with progress
- [x] Frontend polls task status correctly
- [x] Modal updates progress bars in real-time
- [x] Modal shows generated image thumbnails
- [x] Modal handles errors per image
- [x] Modal shows completion summary
- [x] Images list reloads after modal close
---
## Notes
- Images are processed **sequentially** (one at a time) to respect API rate limits
- Each image failure is handled independently (doesn't stop other images)
- Progress updates are sent via Celery task meta (polled every 1 second)
- Task status endpoint already exists and is reused
- Integration settings are read from database (not hardcoded)
- Prompt templates support fallback if formatting fails
- Image URLs are saved directly to Images model
- Status is updated to 'generated' or 'failed' per image
---
**End of Stage 2 & 3 Changelog**

841
docs/IMAGE_GENERATION_IMPLEMENTATION_PLAN.md
View File

@@ -1,841 +0,0 @@
# Image Generation Implementation Plan
## Complete Plan for Generating Images from Prompts
**Date:** 2025-01-XX
**Scope:** Implement image generation AI function following existing AI framework patterns
---
## Table of Contents
1. [System Understanding](#1-system-understanding)
2. [Architecture Overview](#2-architecture-overview)
3. [Implementation Plan](#3-implementation-plan)
4. [Technical Details](#4-technical-details)
5. [Frontend Integration](#5-frontend-integration)
6. [Testing Strategy](#6-testing-strategy)
---
## 1. System Understanding
### 1.1 Current AI Framework Architecture
The system uses a unified AI framework with the following components:
**Core Flow:**
```
Frontend API Call
↓
views.py (@action endpoint)
↓
run_ai_task (ai/tasks.py) - Unified Celery task entrypoint
↓
AIEngine (ai/engine.py) - Orchestrator (6 phases: INIT, PREP, AI_CALL, PARSE, SAVE, DONE)
↓
BaseAIFunction implementation
↓
AICore (ai/ai_core.py) - Centralized AI request handler
↓
AI Provider (OpenAI/Runware)
```
**Existing AI Functions:**
1. **AutoClusterFunction** (`auto_cluster.py`) - Groups keywords into clusters
2. **GenerateIdeasFunction** (`generate_ideas.py`) - Generates content ideas from clusters
3. **GenerateContentFunction** (`generate_content.py`) - Generates article content from ideas
4. **GenerateImagePromptsFunction** (`generate_image_prompts.py`) - Extracts image prompts from content
**Key Components:**
- **BaseAIFunction** - Abstract base class with methods: `get_name()`, `validate()`, `prepare()`, `build_prompt()`, `parse_response()`, `save_output()`
- **AIEngine** - Manages lifecycle, progress tracking, cost tracking, error handling
- **PromptRegistry** - Centralized prompt management with hierarchy (task → DB → default)
- **AICore** - Handles API calls to OpenAI/Runware for both text and image generation
- **IntegrationSettings** - Stores account-specific configurations (models, API keys, image settings)
### 1.2 Image Generation System (WordPress Plugin Reference)
**Key Learnings from WP Plugin:**
1. **Queue-Based Processing:**
- Images are processed sequentially in a queue
- Each image has its own progress bar (0-50% in 7s, 50-75% in 5s, 75-95% incrementally)
- Progress modal shows all images being processed with individual status
2. **Image Types:**
- Featured image (1 per content)
- In-article images (configurable: 1-5 per content)
- Desktop images (if enabled)
- Mobile images (if enabled)
3. **Settings from IntegrationSettings:**
- `provider`: 'openai' or 'runware'
- `model`: Model name (e.g., 'dall-e-3', 'runware:97@1')
- `image_type`: 'realistic', 'artistic', 'cartoon'
- `max_in_article_images`: 1-5
- `image_format`: 'webp', 'jpg', 'png'
- `desktop_enabled`: boolean
- `mobile_enabled`: boolean
4. **Prompt Templates:**
- `image_prompt_template`: Template for formatting prompts (uses {post_title}, {image_prompt}, {image_type})
- `negative_prompt`: Negative prompt for Runware (OpenAI doesn't support)
5. **Progress Tracking:**
- Real-time progress updates via Celery
- Individual image status tracking
- Success/failure per image
### 1.3 Current Image Generation Function
**Existing:** `GenerateImagesFunction` (`generate_images.py`)
- **Status:** Partially implemented, uses old pattern
- **Issues:**
- Still references `Tasks` instead of `Content`
- Doesn't follow the new unified framework pattern
- Uses legacy `generate_images_core()` wrapper
- Doesn't properly queue multiple images
**What We Need:**
- New function: `GenerateImagesFromPromptsFunction`
- Should work with `Images` model (which now has `content` relationship)
- Should process images in queue (one at a time)
- Should use progress modal similar to other AI functions
- Should use prompt templates and negative prompts from Thinker/Prompts
---
## 2. Architecture Overview
### 2.1 New Function: `GenerateImagesFromPromptsFunction`
**Purpose:** Generate actual images from existing image prompts stored in `Images` model
**Input:**
- `ids`: List of Image IDs (or Content IDs) to generate images for
- Images must have `prompt` field populated (from `GenerateImagePromptsFunction`)
**Output:**
- Updates `Images` records with:
- `image_url`: Generated image URL
- `status`: 'generated' (or 'failed' on error)
**Flow:**
1. **INIT (0-10%)**: Validate image IDs, check prompts exist
2. **PREP (10-25%)**: Load images, get settings, prepare queue
3. **AI_CALL (25-70%)**: Generate images sequentially (one per AI_CALL phase)
4. **PARSE (70-85%)**: Parse image URLs from responses
5. **SAVE (85-98%)**: Update Images records with URLs
6. **DONE (98-100%)**: Complete
### 2.2 Key Differences from Other Functions
**Unlike text generation functions:**
- **Multiple AI calls**: One AI call per image (not one call for all)
- **Sequential processing**: Images must be generated one at a time (rate limits)
- **Progress per image**: Need to track progress for each individual image
- **Different API**: Uses `AICore.generate_image()` instead of `AICore.run_ai_request()`
**Similarities:**
- Uses same `BaseAIFunction` pattern
- Uses same `AIEngine` orchestrator
- Uses same progress tracking system
- Uses same error handling
---
## 3. Implementation Plan
### Phase 1: Backend AI Function
#### 3.1 Create `GenerateImagesFromPromptsFunction`
**File:** `backend/igny8_core/ai/functions/generate_images_from_prompts.py`
**Class Structure:**
```python
class GenerateImagesFromPromptsFunction(BaseAIFunction):
def get_name(self) -> str:
return 'generate_images_from_prompts'
def get_metadata(self) -> Dict:
return {
'display_name': 'Generate Images from Prompts',
'description': 'Generate actual images from image prompts',
'phases': {
'INIT': 'Validating image prompts...',
'PREP': 'Preparing image generation queue...',
'AI_CALL': 'Generating images with AI...',
'PARSE': 'Processing image URLs...',
'SAVE': 'Saving image URLs...',
'DONE': 'Images generated!'
}
}
def validate(self, payload: dict, account=None) -> Dict:
"""Validate image IDs and check prompts exist"""
# Check for 'ids' array
# Check images exist and have prompts
# Check images have status='pending'
# Check account matches
def prepare(self, payload: dict, account=None) -> Dict:
"""Load images and settings"""
# Load Images records by IDs
# Get IntegrationSettings for image_generation
# Extract: provider, model, image_type, image_format, etc.
# Get prompt templates from PromptRegistry
# Return: {
# 'images': [Image objects],
# 'settings': {...},
# 'image_prompt_template': str,
# 'negative_prompt': str
# }
def build_prompt(self, data: Dict, account=None) -> Dict:
"""Format prompt using template"""
# For each image in queue:
# - Get content title (from image.content)
# - Format prompt using image_prompt_template
# - Return formatted prompt + image_type
# Note: This is called once per image (AIEngine handles iteration)
def parse_response(self, response: Dict, step_tracker=None) -> Dict:
"""Parse image URL from response"""
# Response from AICore.generate_image() has:
# - 'url': Image URL
# - 'revised_prompt': (optional)
# - 'cost': (optional)
# Return: {'url': str, 'revised_prompt': str, 'cost': float}
def save_output(self, parsed: Dict, original_data: Dict, account=None, ...) -> Dict:
"""Update Images record with URL"""
# Get image from original_data
# Update Images record:
# - image_url = parsed['url']
# - status = 'generated'
# - updated_at = now()
# Return: {'count': 1, 'images_generated': 1}
```
**Key Implementation Details:**
1. **Multiple AI Calls Handling:**
- `AIEngine` will call `build_prompt()` → `AI_CALL` → `parse_response()` → `SAVE` for each image
- Need to track which image is being processed
- Use `step_tracker` to log progress per image
2. **Prompt Formatting:**
```python
# Get template from PromptRegistry
template = PromptRegistry.get_image_prompt_template(account)
# Format with content title and prompt
formatted = template.format(
post_title=image.content.title or image.content.meta_title,
image_prompt=image.prompt,
image_type=settings['image_type']
)
```
3. **Image Generation:**
```python
# Use AICore.generate_image()
result = ai_core.generate_image(
prompt=formatted_prompt,
provider=settings['provider'],
model=settings['model'],
size='1024x1024', # Default or from settings
negative_prompt=negative_prompt if provider == 'runware' else None,
function_name='generate_images_from_prompts'
)
```
4. **Progress Tracking:**
- Track total images: `len(images)`
- Track completed: Increment after each SAVE
- Update progress: `(completed / total) * 100`
#### 3.2 Update AIEngine for Multiple AI Calls
**File:** `backend/igny8_core/ai/engine.py`
**Changes Needed:**
- Detect if function needs multiple AI calls (check function name or metadata)
- For `generate_images_from_prompts`:
- Loop through images in PREP data
- For each image:
- Call `build_prompt()` with single image
- Call `AI_CALL` phase (generate image)
- Call `parse_response()`
- Call `SAVE` phase
- Update progress: `(current_image / total_images) * 100`
- After all images: Call DONE phase
**Alternative Approach (Simpler):**
- Process all images in `save_output()` method
- Make AI calls directly in `save_output()` (not through AIEngine phases)
- Update progress manually via `progress_tracker.update()`
- This is simpler but less consistent with framework
**Recommended Approach:**
- Use AIEngine's phase system
- Add metadata flag: `requires_multiple_ai_calls: True`
- AIEngine detects this and loops through items
#### 3.3 Register Function
**File:** `backend/igny8_core/ai/registry.py`
```python
def _load_generate_images_from_prompts():
from igny8_core.ai.functions.generate_images_from_prompts import GenerateImagesFromPromptsFunction
return GenerateImagesFromPromptsFunction
register_lazy_function('generate_images_from_prompts', _load_generate_images_from_prompts)
```
**File:** `backend/igny8_core/ai/functions/__init__.py`
```python
from .generate_images_from_prompts import GenerateImagesFromPromptsFunction
__all__ = [
...
'GenerateImagesFromPromptsFunction',
]
```
#### 3.4 Add Model Configuration
**File:** `backend/igny8_core/ai/settings.py`
```python
MODEL_CONFIG = {
...
'generate_images_from_prompts': {
'model': 'dall-e-3', # Default, overridden by IntegrationSettings
'max_tokens': None, # Not used for images
'temperature': None, # Not used for images
'response_format': None, # Not used for images
},
}
FUNCTION_TO_PROMPT_TYPE = {
...
'generate_images_from_prompts': None, # Uses image_prompt_template, not text prompt
}
```
#### 3.5 Update Progress Messages
**File:** `backend/igny8_core/ai/engine.py`
```python
def _get_prep_message(self, function_name: str, count: int, data: Any) -> str:
...
elif function_name == 'generate_images_from_prompts':
total_images = len(data.get('images', []))
return f"Preparing to generate {total_images} image{'s' if total_images != 1 else ''}"
def _get_ai_call_message(self, function_name: str, count: int) -> str:
...
elif function_name == 'generate_images_from_prompts':
return f"Generating image {count} of {total} with AI"
def _get_parse_message_with_count(self, function_name: str, count: int) -> str:
...
elif function_name == 'generate_images_from_prompts':
return f"{count} image{'s' if count != 1 else ''} generated"
def _get_save_message(self, function_name: str, count: int) -> str:
...
elif function_name == 'generate_images_from_prompts':
return f"Saving {count} image{'s' if count != 1 else ''}"
```
### Phase 2: API Endpoint
#### 3.6 Add API Endpoint
**File:** `backend/igny8_core/modules/writer/views.py`
**Add to `ImagesViewSet`:**
```python
@action(detail=False, methods=['post'], url_path='generate_images', url_name='generate_images')
def generate_images(self, request):
"""Generate images from prompts for image records"""
from igny8_core.ai.tasks import run_ai_task
account = getattr(request, 'account', None)
ids = request.data.get('ids', [])
if not ids:
return Response({
'error': 'No IDs provided',
'type': 'ValidationError'
}, status=status.HTTP_400_BAD_REQUEST)
account_id = account.id if account else None
# Queue Celery task
try:
if hasattr(run_ai_task, 'delay'):
task = run_ai_task.delay(
function_name='generate_images_from_prompts',
payload={'ids': ids},
account_id=account_id
)
return Response({
'success': True,
'task_id': str(task.id),
'message': 'Image generation started'
}, status=status.HTTP_200_OK)
else:
# Fallback to synchronous execution
result = run_ai_task(
function_name='generate_images_from_prompts',
payload={'ids': ids},
account_id=account_id
)
if result.get('success'):
return Response({
'success': True,
'images_generated': result.get('count', 0),
'message': 'Images generated successfully'
}, status=status.HTTP_200_OK)
else:
return Response({
'error': result.get('error', 'Image generation failed'),
'type': 'TaskExecutionError'
}, status=status.HTTP_500_INTERNAL_SERVER_ERROR)
except Exception as e:
return Response({
'error': str(e),
'type': 'ExecutionError'
}, status=status.HTTP_500_INTERNAL_SERVER_ERROR)
```
### Phase 3: Frontend Integration
#### 3.7 Add API Function
**File:** `frontend/src/services/api.ts`
```typescript
export async function generateImages(imageIds: number[]): Promise<any> {
return fetchAPI('/v1/writer/images/generate_images/', {
method: 'POST',
body: JSON.stringify({ ids: imageIds }),
});
}
```
#### 3.8 Add Generate Images Button
**File:** `frontend/src/config/pages/images.config.tsx`
**Add to row actions or status column:**
- Add "Generate Images" button in status column
- Only show if status is 'pending' and prompt exists
- Button should trigger generation for all images for that content
**File:** `frontend/src/pages/Writer/Images.tsx`
**Add handler:**
```typescript
const handleGenerateImages = useCallback(async (contentId: number) => {
try {
// Get all pending images for this content
const contentImages = images.find(g => g.content_id === contentId);
if (!contentImages) return;
// Collect all image IDs with prompts
const imageIds: number[] = [];
if (contentImages.featured_image?.id && contentImages.featured_image.status === 'pending') {
imageIds.push(contentImages.featured_image.id);
}
contentImages.in_article_images.forEach(img => {
if (img.id && img.status === 'pending' && img.prompt) {
imageIds.push(img.id);
}
});
if (imageIds.length === 0) {
toast.info('No pending images with prompts found');
return;
}
const result = await generateImages(imageIds);
if (result.success) {
if (result.task_id) {
// Open progress modal
progressModal.openModal(
result.task_id,
'Generate Images',
'ai-generate-images-from-prompts-01-desktop'
);
} else {
toast.success(`Images generated: ${result.images_generated || 0} image${(result.images_generated || 0) === 1 ? '' : 's'} created`);
loadImages();
}
} else {
toast.error(result.error || 'Failed to generate images');
}
} catch (error: any) {
toast.error(`Failed to generate images: ${error.message}`);
}
}, [toast, progressModal, loadImages, images]);
```
#### 3.9 Update Progress Modal
**File:** `frontend/src/components/common/ProgressModal.tsx`
**Add support for image generation:**
- Update step labels for `generate_images_from_prompts`
- Show progress per image
- Display generated images in modal (optional, like WP plugin)
**Step Labels:**
```typescript
if (funcName.includes('generate_images_from_prompts')) {
return [
{ phase: 'INIT', label: 'Validating image prompts' },
{ phase: 'PREP', label: 'Preparing image generation queue' },
{ phase: 'AI_CALL', label: 'Generating images with AI' },
{ phase: 'PARSE', label: 'Processing image URLs' },
{ phase: 'SAVE', label: 'Saving image URLs' },
];
}
```
**Success Message:**
```typescript
if (funcName.includes('generate_images_from_prompts')) {
const imageCount = extractCount(/(\d+)\s+image/i, stepLogs || []);
if (imageCount) {
return `${imageCount} image${imageCount !== '1' ? 's' : ''} generated successfully`;
}
return 'Images generated successfully';
}
```
---
## 4. Technical Details
### 4.1 Image Generation API
**AICore.generate_image()** already exists and handles:
- OpenAI DALL-E (dall-e-2, dall-e-3)
- Runware API
- Negative prompts (Runware only)
- Cost tracking
- Error handling
**Usage:**
```python
result = ai_core.generate_image(
prompt=formatted_prompt,
provider='openai', # or 'runware'
model='dall-e-3', # or 'runware:97@1'
size='1024x1024',
negative_prompt=negative_prompt, # Only for Runware
function_name='generate_images_from_prompts'
)
```
**Response:**
```python
{
'url': 'https://...', # Image URL
'revised_prompt': '...', # OpenAI may revise prompt
'cost': 0.04, # Cost in USD
'error': None # Error message if failed
}
```
### 4.2 Settings Retrieval
**From IntegrationSettings:**
```python
integration = IntegrationSettings.objects.get(
account=account,
integration_type='image_generation',
is_active=True
)
config = integration.config
provider = config.get('provider') or config.get('service', 'openai')
if provider == 'runware':
model = config.get('model') or config.get('runwareModel', 'runware:97@1')
else:
model = config.get('model', 'dall-e-3')
image_type = config.get('image_type', 'realistic')
image_format = config.get('image_format', 'webp')
```
### 4.3 Prompt Templates
**From PromptRegistry:**
```python
image_prompt_template = PromptRegistry.get_image_prompt_template(account)
negative_prompt = PromptRegistry.get_negative_prompt(account)
```
**Formatting:**
```python
formatted = image_prompt_template.format(
post_title=content.title or content.meta_title,
image_prompt=image.prompt,
image_type=image_type # 'realistic', 'artistic', 'cartoon'
)
```
### 4.4 Error Handling
**Per-Image Errors:**
- If one image fails, continue with others
- Mark failed image: `status='failed'`
- Log error in `Images` record or separate error field
- Return success with partial count: `{'success': True, 'images_generated': 3, 'images_failed': 1}`
**Validation Errors:**
- No prompts: Skip image, log warning
- No settings: Return error, don't start generation
- Invalid provider/model: Return error
---
## 5. Frontend Integration
### 5.1 Images Page Updates
**File:** `frontend/src/pages/Writer/Images.tsx`
**Changes:**
1. Add "Generate Images" button in status column (or row actions)
2. Button only enabled if:
- Status is 'pending'
- Prompt exists
- Content has at least one pending image
3. On click: Collect all pending image IDs for that content
4. Call API: `generateImages(imageIds)`
5. Open progress modal if async
6. Reload images on completion
### 5.2 Progress Modal Updates
**File:** `frontend/src/components/common/ProgressModal.tsx`
**Changes:**
1. Add step definitions for `generate_images_from_prompts`
2. Update progress messages
3. Show image count in messages
4. Optional: Display generated images in modal (like WP plugin)
### 5.3 Table Actions Config
**File:** `frontend/src/config/pages/table-actions.config.tsx`
**Add row action (optional):**
```typescript
'/writer/images': {
rowActions: [
{
key: 'generate_images',
label: 'Generate Images',
icon: <BoltIcon className="w-5 h-5" />,
variant: 'primary',
},
],
}
```
---
## 6. Testing Strategy
### 6.1 Unit Tests
**Test Function Methods:**
- `validate()`: Test with valid/invalid IDs, missing prompts, wrong status
- `prepare()`: Test settings retrieval, prompt template loading
- `build_prompt()`: Test prompt formatting
- `parse_response()`: Test URL extraction
- `save_output()`: Test Images record update
### 6.2 Integration Tests
**Test Full Flow:**
1. Create Images records with prompts
2. Call API endpoint
3. Verify Celery task created
4. Verify progress updates
5. Verify Images records updated with URLs
6. Verify status changed to 'generated'
### 6.3 Error Scenarios
**Test:**
- Missing IntegrationSettings
- Invalid provider/model
- API errors (rate limits, invalid API key)
- Partial failures (some images succeed, some fail)
- Missing prompts
- Invalid image IDs
---
## 7. Implementation Checklist
### Backend
- [ ] Create `GenerateImagesFromPromptsFunction` class
- [ ] Implement `validate()` method
- [ ] Implement `prepare()` method
- [ ] Implement `build_prompt()` method
- [ ] Implement `parse_response()` method
- [ ] Implement `save_output()` method
- [ ] Register function in `registry.py`
- [ ] Add to `__init__.py` exports
- [ ] Add model config in `settings.py`
- [ ] Update `AIEngine` progress messages
- [ ] Add API endpoint in `ImagesViewSet`
- [ ] Test with OpenAI provider
- [ ] Test with Runware provider
- [ ] Test error handling
### Frontend
- [ ] Add `generateImages()` API function
- [ ] Add "Generate Images" button to Images page
- [ ] Add click handler
- [ ] Integrate progress modal
- [ ] Update progress modal step labels
- [ ] Update success messages
- [ ] Test UI flow
- [ ] Test error handling
### Documentation
- [ ] Update AI_MASTER_ARCHITECTURE.md
- [ ] Add function to AI_FUNCTIONS_AUDIT_REPORT.md
- [ ] Document API endpoint
- [ ] Document settings requirements
---
## 8. Key Considerations
### 8.1 Rate Limiting
**Issue:** Image generation APIs have rate limits
**Solution:** Process images sequentially (one at a time)
**Implementation:** AIEngine loops through images, waits for each to complete
### 8.2 Cost Tracking
**Issue:** Need to track costs per image
**Solution:** AICore already tracks costs, store in AITaskLog
**Implementation:** Cost is returned from `generate_image()`, log in step_tracker
### 8.3 Progress Updates
**Issue:** Need granular progress (per image)
**Solution:** Update progress after each image: `(completed / total) * 100`
**Implementation:** Track in `save_output()`, update via `progress_tracker.update()`
### 8.4 Error Recovery
**Issue:** If one image fails, should continue with others
**Solution:** Catch errors per image, mark as failed, continue
**Implementation:** Try-catch in `save_output()` per image
### 8.5 Image Display
**Issue:** Should show generated images in progress modal?
**Solution:** Optional enhancement, can add later
**Implementation:** Store image URLs in step logs, display in modal
---
## 9. Alternative Approaches Considered
### 9.1 Process All in save_output()
**Pros:**
- Simpler implementation
- Direct control over loop
**Cons:**
- Doesn't use AIEngine phases properly
- Harder to track progress per image
- Less consistent with framework
**Decision:** Use AIEngine phases with loop detection
### 9.2 Separate Function Per Image
**Pros:**
- Each image is independent task
- Better error isolation
**Cons:**
- Too many Celery tasks
- Harder to track overall progress
- More complex frontend
**Decision:** Single function processes all images sequentially
---
## 10. Success Criteria
✅ Function follows `BaseAIFunction` pattern
✅ Uses `AIEngine` orchestrator
✅ Integrates with progress modal
✅ Uses prompt templates from Thinker/Prompts
✅ Uses settings from IntegrationSettings
✅ Handles errors gracefully
✅ Tracks progress per image
✅ Updates Images records correctly
✅ Works with both OpenAI and Runware
✅ Frontend button triggers generation
✅ Progress modal shows correct steps
✅ Success message shows image count
---
## 11. Next Steps
1. **Start with Backend Function**
- Create `GenerateImagesFromPromptsFunction`
- Implement all methods
- Test with single image
2. **Add API Endpoint**
- Add to `ImagesViewSet`
- Test endpoint
3. **Frontend Integration**
- Add button
- Add handler
- Test flow
4. **Progress Modal**
- Update step labels
- Test progress updates
5. **Error Handling**
- Test error scenarios
- Verify graceful failures
6. **Documentation**
- Update architecture docs
- Add API docs
---
**End of Plan**

1153
docs/MASTER-ARCHITECTURE.md
View File

File diff suppressed because it is too large Load Diff

146
docs/README.md Normal file
View File

@@ -0,0 +1,146 @@
# IGNY8 Documentation
**Last Updated:** 2025-01-XX
**Purpose:** Complete documentation index for the IGNY8 platform.
---
## 📚 Documentation Structure
All documentation is organized in this single folder with the following structure:
### Core Documentation
1. **[01-ARCHITECTURE-TECH-STACK.md](./01-ARCHITECTURE-TECH-STACK.md)**
- Technology stack overview
- System architecture principles
- Infrastructure components
- External service integrations
2. **[02-APP-ARCHITECTURE.md](./02-APP-ARCHITECTURE.md)**
- IGNY8 application architecture
- System hierarchy and relationships
- Module organization
- Complete workflows
- Data flow and processing
- Multi-tenancy architecture
- Security architecture
3. **[03-FRONTEND.md](./03-FRONTEND.md)**
- Frontend architecture
- Project structure
- Routing system
- Template system
- Component library
- State management
- API integration
- Configuration system
- All pages and features
4. **[04-BACKEND.md](./04-BACKEND.md)**
- Backend architecture
- Project structure
- Models and relationships
- ViewSets and API endpoints
- Serializers
- Celery tasks
- Middleware
- All modules (Planner, Writer, System, Billing, Auth)
5. **[05-AI-FUNCTIONS.md](./05-AI-FUNCTIONS.md)**
- AI framework architecture
- All 5 AI functions (complete details)
- AI function execution flow
- Prompt management
- Model configuration
- Progress tracking
- Cost tracking
6. **[06-CHANGELOG.md](./06-CHANGELOG.md)**
- System changelog
- Feature additions
- Updates and improvements
- Version history
---
## 🚀 Quick Start
1. **New to IGNY8?** Start with [01-ARCHITECTURE-TECH-STACK.md](./01-ARCHITECTURE-TECH-STACK.md) for technology overview
2. **Understanding the System?** Read [02-APP-ARCHITECTURE.md](./02-APP-ARCHITECTURE.md) for complete architecture
3. **Frontend Development?** See [03-FRONTEND.md](./03-FRONTEND.md) for all frontend details
4. **Backend Development?** See [04-BACKEND.md](./04-BACKEND.md) for all backend details
5. **Working with AI?** See [05-AI-FUNCTIONS.md](./05-AI-FUNCTIONS.md) for AI functions
6. **What's New?** Check [06-CHANGELOG.md](./06-CHANGELOG.md) for recent changes
---
## 📋 Documentation Overview
### System Capabilities
- **Multi-Tenancy**: Complete account isolation with automatic filtering
- **Planner Module**: Keywords, Clusters, Content Ideas management
- **Writer Module**: Tasks, Content, Images generation and management
- **Thinker Module**: Prompts, Author Profiles, Strategies, Image Testing
- **System Module**: Settings, Integrations, AI Prompts
- **Billing Module**: Credits, Transactions, Usage Logs
- **AI Functions**: 5 AI operations (Auto Cluster, Generate Ideas, Generate Content, Generate Image Prompts, Generate Images)
### Technology Stack
- **Backend**: Django 5.2+ with Django REST Framework
- **Frontend**: React 19 with TypeScript and Vite
- **Database**: PostgreSQL 15
- **Task Queue**: Celery with Redis
- **Deployment**: Docker-based containerization
- **Reverse Proxy**: Caddy (HTTPS)
---
## 🔍 Finding Information
### By Topic
- **Architecture & Design**: [01-ARCHITECTURE-TECH-STACK.md](./01-ARCHITECTURE-TECH-STACK.md), [02-APP-ARCHITECTURE.md](./02-APP-ARCHITECTURE.md)
- **Frontend Development**: [03-FRONTEND.md](./03-FRONTEND.md)
- **Backend Development**: [04-BACKEND.md](./04-BACKEND.md)
- **AI Functions**: [05-AI-FUNCTIONS.md](./05-AI-FUNCTIONS.md)
- **Changes & Updates**: [06-CHANGELOG.md](./06-CHANGELOG.md)
### By Module
- **Planner**: See [02-APP-ARCHITECTURE.md](./02-APP-ARCHITECTURE.md) (Module Organization) and [04-BACKEND.md](./04-BACKEND.md) (Planner Module)
- **Writer**: See [02-APP-ARCHITECTURE.md](./02-APP-ARCHITECTURE.md) (Module Organization) and [04-BACKEND.md](./04-BACKEND.md) (Writer Module)
- **Thinker**: See [03-FRONTEND.md](./03-FRONTEND.md) (Thinker Pages) and [04-BACKEND.md](./04-BACKEND.md) (System Module)
- **System**: See [04-BACKEND.md](./04-BACKEND.md) (System Module)
- **Billing**: See [04-BACKEND.md](./04-BACKEND.md) (Billing Module)
---
## 📝 Documentation Standards
- **No Code**: Documentation focuses on workflows, features, and architecture (no code snippets)
- **Complete**: All workflows and features are documented
- **Accurate**: Documentation reflects current system state
- **Detailed**: Comprehensive coverage of all aspects
---
## 🔄 Keeping Documentation Updated
Documentation is updated when:
- New features are added
- Workflows change
- Architecture evolves
- Modules are modified
**Last Review**: 2025-01-XX
**Next Review**: As system evolves
---
## 📞 Support
For questions or clarifications about the documentation, refer to the specific document or contact the development team.

137
docs/ai-docs/AI_FILES_ANALYSIS.md
View File

@@ -1,137 +0,0 @@
# AI Folder Files Analysis
## Which Files Are Actually Needed for 3 Active Functions
### ✅ **REQUIRED FILES** (All needed for active functions)
#### 1. **tasks.py** - ✅ REQUIRED
- **Purpose:** Unified Celery task entrypoint
- **Used by:** All views (planner/views.py, writer/views.py)
- **Dependencies:** engine.py, registry.py
- **Status:** KEEP
#### 2. **engine.py** - ✅ REQUIRED
- **Purpose:** Central orchestrator for all AI functions
- **Used by:** tasks.py
- **Dependencies:** base.py, tracker.py, ai_core.py, settings.py, models.py
- **Status:** KEEP
#### 3. **base.py** - ✅ REQUIRED
- **Purpose:** Abstract base class for all AI functions
- **Used by:** All function classes (AutoClusterFunction, GenerateIdeasFunction, GenerateContentFunction)
- **Dependencies:** None
- **Status:** KEEP
#### 4. **registry.py** - ✅ REQUIRED
- **Purpose:** Function registry with lazy loading
- **Used by:** tasks.py
- **Dependencies:** base.py, function modules
- **Status:** KEEP
#### 5. **ai_core.py** - ✅ REQUIRED
- **Purpose:** Centralized AI request handler
- **Used by:** engine.py, all function classes
- **Dependencies:** constants.py, tracker.py
- **Status:** KEEP
#### 6. **prompts.py** - ✅ REQUIRED
- **Purpose:** Centralized prompt management
- **Used by:** All function classes (build_prompt methods)
- **Dependencies:** None
- **Status:** KEEP
#### 7. **settings.py** - ✅ REQUIRED
- **Purpose:** Model configurations per function
- **Used by:** engine.py
- **Dependencies:** constants.py (via ai_processor import)
- **Status:** KEEP
#### 8. **validators.py** - ✅ REQUIRED
- **Purpose:** Validation functions
- **Used by:** All function classes (validate methods)
- **Dependencies:** constants.py, models
- **Status:** KEEP
#### 9. **tracker.py** - ✅ REQUIRED
- **Purpose:** Progress tracking utilities
- **Used by:** engine.py, ai_core.py
- **Dependencies:** types.py (imported but NOT USED), constants.py
- **Status:** KEEP (but can remove types.py import)
#### 10. **constants.py** - ✅ REQUIRED
- **Purpose:** AI constants (model rates, valid models)
- **Used by:** ai_core.py, settings.py, validators.py
- **Dependencies:** None
- **Status:** KEEP
#### 11. **models.py** - ✅ REQUIRED
- **Purpose:** AITaskLog database model
- **Used by:** engine.py (for logging), admin.py
- **Dependencies:** AccountBaseModel
- **Status:** KEEP
#### 12. **apps.py** - ✅ REQUIRED
- **Purpose:** Django app configuration
- **Used by:** Django (app registration)
- **Dependencies:** admin.py
- **Status:** KEEP
#### 13. **admin.py** - ✅ REQUIRED (Optional but recommended)
- **Purpose:** Django admin interface for AITaskLog
- **Used by:** apps.py (auto-imported)
- **Dependencies:** models.py
- **Status:** KEEP (useful for debugging/admin)
---
### ❌ **UNUSED FILES** (Can be removed)
#### 1. **types.py** - ❌ NOT USED
- **Purpose:** Type definitions (StepLog, ProgressState, AITaskResult dataclasses)
- **Used by:** tracker.py (imported but never instantiated)
- **Actual usage:** None - tracker.py uses plain Dict types instead
- **Status:** **SAFE TO REMOVE**
- **Action:** Remove file and remove import from tracker.py
---
## Summary
### Files to KEEP (13 files):
1. ✅ tasks.py
2. ✅ engine.py
3. ✅ base.py
4. ✅ registry.py
5. ✅ ai_core.py
6. ✅ prompts.py
7. ✅ settings.py
8. ✅ validators.py
9. ✅ tracker.py
10. ✅ constants.py
11. ✅ models.py
12. ✅ apps.py
13. ✅ admin.py
### Files to REMOVE (1 file):
1. ❌ **types.py** - Imported but never used
---
## Action Items
1. **Remove types.py** - File is not used
2. **Remove import from tracker.py** - Remove `from igny8_core.ai.types import StepLog, ProgressState` (line 8)
---
## Verification
All other files are actively used in the execution flow:
- **tasks.py** → **registry.py** → **engine.py** → **base.py** (function classes)
- **engine.py** → **ai_core.py** → **constants.py**
- **engine.py** → **tracker.py** → **constants.py**
- **engine.py** → **settings.py** → **constants.py**
- **engine.py** → **models.py** (AITaskLog)
- **function classes** → **prompts.py**, **validators.py**
- **apps.py** → **admin.py** → **models.py**
1

277
docs/ai-docs/AI_FUNCTIONS_AUDIT_REPORT.md
View File

@@ -1,277 +0,0 @@
# AI Functions Deep Audit Report
## Clustering, Idea Generation, and Content Generation
**Date:** 2025-01-XX
**Scope:** Complete audit of AI functions for clustering, idea generation, and content generation to identify unused code and files safe to remove.
---
## Executive Summary
This audit identifies **legacy code, deprecated functions, and unused implementations** that are safe to remove from the codebase. The current system uses a unified AI framework through `run_ai_task` → `AIEngine` → `BaseAIFunction` implementations.
---
## 1. CURRENT ACTIVE ARCHITECTURE
### 1.1 Active Flow (What's Actually Used)
```
Frontend API Call
↓
views.py (auto_cluster/auto_generate_ideas/auto_generate_content)
↓
run_ai_task (ai/tasks.py) - Unified Celery task entrypoint
↓
AIEngine (ai/engine.py) - Orchestrator
↓
BaseAIFunction implementations:
- AutoClusterFunction (ai/functions/auto_cluster.py)
- GenerateIdeasFunction (ai/functions/generate_ideas.py)
- GenerateContentFunction (ai/functions/generate_content.py)
↓
AICore (ai/ai_core.py) - Centralized AI request handler
↓
AIProvider (OpenAI/Runware)
```
### 1.2 Active Files (KEEP)
#### Core Framework
- ✅ `backend/igny8_core/ai/tasks.py` - Unified Celery task (`run_ai_task`)
- ✅ `backend/igny8_core/ai/engine.py` - AIEngine orchestrator
- ✅ `backend/igny8_core/ai/base.py` - BaseAIFunction abstract class
- ✅ `backend/igny8_core/ai/ai_core.py` - AICore centralized handler
- ✅ `backend/igny8_core/ai/registry.py` - Function registry
- ✅ `backend/igny8_core/ai/prompts.py` - PromptRegistry
- ✅ `backend/igny8_core/ai/settings.py` - Model configurations
- ✅ `backend/igny8_core/ai/constants.py` - AI constants
- ✅ `backend/igny8_core/ai/tracker.py` - Progress tracking
- ✅ `backend/igny8_core/ai/validators.py` - Validation functions
#### Function Implementations
- ✅ `backend/igny8_core/ai/functions/auto_cluster.py` - AutoClusterFunction
- ✅ `backend/igny8_core/ai/functions/generate_ideas.py` - GenerateIdeasFunction
- ✅ `backend/igny8_core/ai/functions/generate_content.py` - GenerateContentFunction
- ✅ `backend/igny8_core/ai/functions/__init__.py` - Exports
#### API Endpoints
- ✅ `backend/igny8_core/modules/planner/views.py` - KeywordViewSet.auto_cluster(), ClusterViewSet.auto_generate_ideas()
- ✅ `backend/igny8_core/modules/writer/views.py` - TasksViewSet.auto_generate_content()
---
## 2. DEPRECATED / UNUSED CODE (SAFE TO REMOVE)
### 2.1 Legacy Wrapper Functions (NOT USED)
#### ❌ `generate_ideas_core()` - **SAFE TO REMOVE**
- **Location:** `backend/igny8_core/ai/functions/generate_ideas.py:234`
- **Status:** Legacy wrapper function for backward compatibility
- **Usage:** ❌ **NOT CALLED ANYWHERE** (only in commented test code)
- **Purpose:** Was meant for direct calls without Celery, but all calls now go through `run_ai_task`
- **Action:** **REMOVE** - Function and export from `__init__.py`
#### ❌ `generate_content_core()` - **SAFE TO REMOVE**
- **Location:** `backend/igny8_core/ai/functions/generate_content.py:303`
- **Status:** Legacy wrapper function for backward compatibility
- **Usage:** ❌ **NOT CALLED ANYWHERE** (only in commented test code)
- **Purpose:** Was meant for direct calls without Celery, but all calls now go through `run_ai_task`
- **Action:** **REMOVE** - Function and export from `__init__.py`
### 2.2 AIProcessor Deprecated Methods (NOT USED)
#### ❌ `AIProcessor.cluster_keywords()` - **SAFE TO REMOVE**
- **Location:** `backend/igny8_core/utils/ai_processor.py:1049-1282`
- **Status:** ⚠️ **DEPRECATED** (marked with deprecation warning)
- **Usage:** ❌ **NOT CALLED ANYWHERE**
- **Lines:** ~233 lines of code
- **Action:** **REMOVE** entire method
#### ❌ `AIProcessor.generate_ideas()` - **SAFE TO REMOVE**
- **Location:** `backend/igny8_core/utils/ai_processor.py:1284-1363`
- **Status:** Legacy method
- **Usage:** ❌ **NOT CALLED ANYWHERE**
- **Lines:** ~80 lines of code
- **Action:** **REMOVE** entire method
#### ❌ `AIProcessor.generate_content()` - **SAFE TO REMOVE**
- **Location:** `backend/igny8_core/utils/ai_processor.py:433-531`
- **Status:** Legacy method
- **Usage:** ❌ **NOT CALLED ANYWHERE** (only called internally by `extract_image_prompts`, which is also unused)
- **Lines:** ~98 lines of code
- **Action:** **REMOVE** entire method
#### ❌ `AIProcessor.extract_image_prompts()` - **SAFE TO REMOVE**
- **Location:** `backend/igny8_core/utils/ai_processor.py:471-580`
- **Status:** Legacy method
- **Usage:** ❌ **NOT CALLED ANYWHERE** (new framework uses `GenerateImagesFunction` which uses `AICore.run_ai_request` directly)
- **Lines:** ~110 lines of code
- **Action:** **REMOVE** entire method
**Note:** `AIProcessor.generate_image()` is **STILL USED** in `integration_views.py` for image generation, so keep the class and that method.
### 2.3 Unused Exports
#### ❌ `generate_ideas_core` export - **SAFE TO REMOVE**
- **Location:** `backend/igny8_core/ai/functions/__init__.py:5,12`
- **Action:** Remove from imports and `__all__`
#### ❌ `generate_content_core` export - **SAFE TO REMOVE**
- **Location:** `backend/igny8_core/ai/functions/__init__.py:6,14`
- **Action:** Remove from imports and `__all__`
### 2.4 Test File Cleanup
#### ⚠️ `backend/igny8_core/ai/tests/test_run.py`
- **Status:** Contains commented-out code referencing removed functions
- **Action:** Clean up commented code (lines 16, 63, 75, 126-127)
---
## 3. AIProcessor - PARTIAL CLEANUP
### 3.1 What to KEEP in AIProcessor
✅ **KEEP:**
- `AIProcessor.__init__()` - Initialization
- `AIProcessor._get_api_key()` - API key retrieval
- `AIProcessor._get_model()` - Model retrieval
- `AIProcessor._call_openai()` - OpenAI API calls (used by generate_image)
- `AIProcessor._extract_json_from_response()` - JSON extraction
- `AIProcessor.generate_image()` - **STILL USED** in `integration_views.py`
- `AIProcessor.get_prompt()` - May be used by generate_image
- Constants imports (MODEL_RATES, etc.) - Used by AICore
### 3.2 What to REMOVE from AIProcessor
❌ **REMOVE:**
- `AIProcessor.cluster_keywords()` - ~233 lines
- `AIProcessor.generate_ideas()` - ~80 lines
- `AIProcessor.generate_content()` - ~98 lines
- `AIProcessor.extract_image_prompts()` - ~110 lines
- **Total:** ~521 lines of unused code
---
## 4. FILES ALREADY DELETED (Good!)
✅ **Already Removed:**
- `backend/igny8_core/modules/planner/tasks.py` - ✅ Already deleted
- `backend/igny8_core/modules/writer/tasks.py` - ✅ Already deleted
---
## 5. SUMMARY OF REMOVALS
### 5.1 Functions to Remove
| Function | Location | Lines | Status |
|----------|----------|-------|--------|
| `generate_ideas_core()` | `ai/functions/generate_ideas.py:234` | ~100 | ❌ Remove |
| `generate_content_core()` | `ai/functions/generate_content.py:303` | ~85 | ❌ Remove |
| `AIProcessor.cluster_keywords()` | `utils/ai_processor.py:1049` | ~233 | ❌ Remove |
| `AIProcessor.generate_ideas()` | `utils/ai_processor.py:1284` | ~80 | ❌ Remove |
| `AIProcessor.generate_content()` | `utils/ai_processor.py:433` | ~98 | ❌ Remove |
| `AIProcessor.extract_image_prompts()` | `utils/ai_processor.py:471` | ~110 | ❌ Remove |
**Total Lines to Remove:** ~706 lines
### 5.2 Exports to Remove
| Export | Location |
|--------|----------|
| `generate_ideas_core` | `ai/functions/__init__.py:5,12` |
| `generate_content_core` | `ai/functions/__init__.py:6,14` |
### 5.3 Test Cleanup
| File | Action |
|------|--------|
| `ai/tests/test_run.py` | Remove commented code (lines 16, 63, 75, 126-127) |
---
## 6. VERIFICATION CHECKLIST
Before removing, verify:
- [ ] ✅ No imports of `generate_ideas_core` or `generate_content_core` anywhere
- [ ] ✅ No calls to `AIProcessor.cluster_keywords()`, `generate_ideas()`, or `generate_content()`
- [ ] ✅ All active code paths use `run_ai_task` → `AIEngine` → `BaseAIFunction`
- [ ] ✅ `AIProcessor.generate_image()` is still used (verify in `integration_views.py`)
- [ ] ✅ Constants from `ai_processor.py` are still imported by `AICore` (verify)
---
## 7. RECOMMENDED REMOVAL ORDER
1. **Phase 1: Remove Legacy Wrapper Functions**
- Remove `generate_ideas_core()` from `generate_ideas.py`
- Remove `generate_content_core()` from `generate_content.py`
- Remove exports from `__init__.py`
2. **Phase 2: Remove AIProcessor Deprecated Methods**
- Remove `AIProcessor.cluster_keywords()`
- Remove `AIProcessor.generate_ideas()`
- Remove `AIProcessor.generate_content()`
- Remove `AIProcessor.extract_image_prompts()` (depends on generate_content, so remove after)
3. **Phase 3: Cleanup Tests**
- Clean up commented code in `test_run.py`
4. **Phase 4: Verification**
- Run full test suite
- Verify all AI functions still work
- Check for any broken imports
---
## 8. IMPACT ANALYSIS
### 8.1 No Breaking Changes Expected
- ✅ All active code paths use the new framework
- ✅ No external dependencies on removed functions
- ✅ Views only use `run_ai_task` (verified)
### 8.2 Benefits
- 🎯 **~706 lines of dead code removed**
- 🎯 **Clearer codebase** - Only active code remains
- 🎯 **Easier maintenance** - No confusion about which path to use
- 🎯 **Reduced technical debt**
---
## 9. FILES TO MODIFY
### 9.1 Files to Edit
1. `backend/igny8_core/ai/functions/generate_ideas.py`
- Remove `generate_ideas_core()` function (lines 234-333)
2. `backend/igny8_core/ai/functions/generate_content.py`
- Remove `generate_content_core()` function (lines 303-386)
3. `backend/igny8_core/ai/functions/__init__.py`
- Remove `generate_ideas_core` import and export
- Remove `generate_content_core` import and export
4. `backend/igny8_core/utils/ai_processor.py`
- Remove `cluster_keywords()` method (lines 1049-1282)
- Remove `generate_ideas()` method (lines 1284-1363)
- Remove `generate_content()` method (lines 433-531)
- Remove `extract_image_prompts()` method (lines 471-580)
5. `backend/igny8_core/ai/tests/test_run.py`
- Remove commented code referencing removed functions
---
## 10. CONCLUSION
This audit identifies **~706 lines of unused legacy code** that can be safely removed without impacting functionality. All active code paths use the unified AI framework (`run_ai_task` → `AIEngine` → `BaseAIFunction`), making these legacy functions obsolete.
**Recommendation:** Proceed with removal in phases as outlined above, with thorough testing after each phase.

802
docs/ai-docs/AI_MASTER_ARCHITECTURE.md
View File

@@ -1,802 +0,0 @@
# AI Master Architecture Document
## Clustering, Idea Generation, and Content Generation
**Version:** 1.0
**Date:** 2025-01-XX
**Scope:** Complete architecture for 3 verified AI functions (clustering, idea generation, content generation)
---
## Table of Contents
1. [Common Architecture](#1-common-architecture)
2. [Auto Cluster Keywords](#2-auto-cluster-keywords)
3. [Generate Ideas](#3-generate-ideas)
4. [Generate Content](#4-generate-content)
---
## 1. Common Architecture
### 1.1 Core Framework Files
#### Entry Point
- **File:** `backend/igny8_core/ai/tasks.py`
- **Function:** `run_ai_task`
- **Purpose:** Unified Celery task entrypoint for all AI functions
- **Parameters:** `function_name` (str), `payload` (dict), `account_id` (int)
- **Flow:** Loads function from registry → Creates AIEngine → Executes function
#### Engine Orchestrator
- **File:** `backend/igny8_core/ai/engine.py`
- **Class:** `AIEngine`
- **Purpose:** Central orchestrator managing lifecycle, progress, logging, cost tracking
- **Methods:**
- `execute` - Main execution pipeline (6 phases: INIT, PREP, AI_CALL, PARSE, SAVE, DONE)
- `_handle_error` - Centralized error handling
- `_log_to_database` - Logs to AITaskLog model
- Helper methods: `_get_input_description`, `_build_validation_message`, `_get_prep_message`, `_get_ai_call_message`, `_get_parse_message`, `_get_parse_message_with_count`, `_get_save_message`, `_calculate_credits_for_clustering`
#### Base Function Class
- **File:** `backend/igny8_core/ai/base.py`
- **Class:** `BaseAIFunction`
- **Purpose:** Abstract base class defining interface for all AI functions
- **Abstract Methods:**
- `get_name` - Returns function name (e.g., 'auto_cluster')
- `prepare` - Loads and prepares data
- `build_prompt` - Builds AI prompt
- `parse_response` - Parses AI response
- `save_output` - Saves results to database
- **Optional Methods:**
- `get_metadata` - Returns display name, description, phases
- `get_max_items` - Returns max items limit (or None)
- `validate` - Validates input payload (default: checks for 'ids')
- `get_model` - Returns model override (default: None, uses account default)
#### Function Registry
- **File:** `backend/igny8_core/ai/registry.py`
- **Functions:**
- `register_function` - Registers function class
- `register_lazy_function` - Registers lazy loader
- `get_function` - Gets function class by name (lazy loads if needed)
- `get_function_instance` - Gets function instance by name
- `list_functions` - Lists all registered functions
- **Lazy Loaders:**
- `_load_auto_cluster` - Loads AutoClusterFunction
- `_load_generate_ideas` - Loads GenerateIdeasFunction
- `_load_generate_content` - Loads GenerateContentFunction
#### AI Core Handler
- **File:** `backend/igny8_core/ai/ai_core.py`
- **Class:** `AICore`
- **Purpose:** Centralized AI request handler for all text generation
- **Methods:**
- `run_ai_request` - Makes API call to OpenAI/Runware
- `extract_json` - Extracts JSON from response (handles markdown code blocks)
#### Prompt Registry
- **File:** `backend/igny8_core/ai/prompts.py`
- **Class:** `PromptRegistry`
- **Purpose:** Centralized prompt management with hierarchical resolution
- **Method:** `get_prompt` - Gets prompt with resolution order:
1. Task-level prompt_override (if exists)
2. DB prompt for (account, function)
3. Default fallback from DEFAULT_PROMPTS registry
- **Prompt Types:**
- `clustering` - For auto_cluster function
- `ideas` - For generate_ideas function
- `content_generation` - For generate_content function
- **Context Placeholders:**
- `[IGNY8_KEYWORDS]` - Replaced with keyword list
- `[IGNY8_CLUSTERS]` - Replaced with cluster list
- `[IGNY8_CLUSTER_KEYWORDS]` - Replaced with cluster keywords
- `[IGNY8_IDEA]` - Replaced with idea data
- `[IGNY8_CLUSTER]` - Replaced with cluster data
- `[IGNY8_KEYWORDS]` - Replaced with keywords (for content)
#### Model Settings
- **File:** `backend/igny8_core/ai/settings.py`
- **Constants:**
- `MODEL_CONFIG` - Model configurations per function (model, max_tokens, temperature, response_format)
- `FUNCTION_ALIASES` - Legacy function name mappings
- **Functions:**
- `get_model_config` - Gets model config for function (reads from IntegrationSettings if account provided)
- `get_model` - Gets model name for function
- `get_max_tokens` - Gets max tokens for function
- `get_temperature` - Gets temperature for function
#### Validators
- **File:** `backend/igny8_core/ai/validators.py`
- **Functions:**
- `validate_ids` - Validates 'ids' array in payload
- `validate_keywords_exist` - Validates keywords exist in database
- `validate_cluster_exists` - Validates cluster exists
- `validate_tasks_exist` - Validates tasks exist
- `validate_cluster_limits` - Validates plan limits (currently disabled - always returns valid)
- `validate_api_key` - Validates API key is configured
- `validate_model` - Validates model is in supported list
- `validate_image_size` - Validates image size for model
#### Progress Tracking
- **File:** `backend/igny8_core/ai/tracker.py`
- **Classes:**
- `StepTracker` - Tracks request/response steps
- `ProgressTracker` - Tracks Celery progress updates
- `CostTracker` - Tracks API costs and tokens
- `ConsoleStepTracker` - Console-based step logging
#### Database Logging
- **File:** `backend/igny8_core/ai/models.py`
- **Model:** `AITaskLog`
- **Fields:** `task_id`, `function_name`, `account`, `phase`, `message`, `status`, `duration`, `cost`, `tokens`, `request_steps`, `response_steps`, `error`, `payload`, `result`
### 1.2 Execution Flow (All Functions)
```
1. API Endpoint (views.py)
↓
2. run_ai_task (tasks.py)
- Gets account from account_id
- Gets function instance from registry
- Creates AIEngine
↓
3. AIEngine.execute (engine.py)
Phase 1: INIT (0-10%)
- Calls function.validate()
- Updates progress tracker
↓
Phase 2: PREP (10-25%)
- Calls function.prepare()
- Calls function.build_prompt()
- Updates progress tracker
↓
Phase 3: AI_CALL (25-70%)
- Gets model config from settings
- Calls AICore.run_ai_request()
- Tracks cost and tokens
- Updates progress tracker
↓
Phase 4: PARSE (70-85%)
- Calls function.parse_response()
- Updates progress tracker
↓
Phase 5: SAVE (85-98%)
- Calls function.save_output()
- Logs credit usage
- Updates progress tracker
↓
Phase 6: DONE (98-100%)
- Logs to AITaskLog
- Returns result
```
---
## 2. Auto Cluster Keywords
### 2.1 Function Implementation
- **File:** `backend/igny8_core/ai/functions/auto_cluster.py`
- **Class:** `AutoClusterFunction`
- **Inherits:** `BaseAIFunction`
### 2.2 API Endpoint
- **File:** `backend/igny8_core/modules/planner/views.py`
- **ViewSet:** `KeywordViewSet`
- **Action:** `auto_cluster`
- **Method:** POST
- **URL Path:** `/v1/planner/keywords/auto_cluster/`
- **Payload:**
- `ids` (list[int]) - Keyword IDs to cluster
- `sector_id` (int, optional) - Sector ID for filtering
- **Response:**
- `success` (bool)
- `task_id` (str) - Celery task ID if async
- `clusters_created` (int) - Number of clusters created
- `keywords_updated` (int) - Number of keywords updated
- `message` (str)
### 2.3 Function Methods
#### `get_name()`
- **Returns:** `'auto_cluster'`
#### `get_metadata()`
- **Returns:** Dict with `display_name`, `description`, `phases` (INIT, PREP, AI_CALL, PARSE, SAVE, DONE)
#### `get_max_items()`
- **Returns:** `None` (no limit)
#### `validate(payload, account)`
- **Validates:**
- Calls `validate_ids` to check for 'ids' array
- Calls `validate_keywords_exist` to verify keywords exist
- **Returns:** Dict with `valid` (bool) and optional `error` (str)
#### `prepare(payload, account)`
- **Loads:**
- Keywords from database (filters by `ids`, `account`, optional `sector_id`)
- Uses `select_related` for: `account`, `site`, `site__account`, `sector`, `sector__site`
- **Returns:** Dict with:
- `keywords` (list[Keyword objects])
- `keyword_data` (list[dict]) - Formatted data with: `id`, `keyword`, `volume`, `difficulty`, `intent`
- `sector_id` (int, optional)
#### `build_prompt(data, account)`
- **Gets Prompt:**
- Calls `PromptRegistry.get_prompt(function_name='auto_cluster', account, context)`
- Context includes: `KEYWORDS` (formatted keyword list), optional `SECTOR` (sector name)
- **Formatting:**
- Formats keywords as: `"- {keyword} (Volume: {volume}, Difficulty: {difficulty}, Intent: {intent})"`
- Replaces `[IGNY8_KEYWORDS]` placeholder
- Adds JSON mode instruction if not present
- **Returns:** Prompt string
#### `parse_response(response, step_tracker)`
- **Parsing:**
- Tries direct JSON parse first
- Falls back to `AICore.extract_json()` if needed (handles markdown code blocks)
- **Extraction:**
- Extracts `clusters` array from JSON
- Handles both dict with 'clusters' key and direct array
- **Returns:** List[Dict] with cluster data:
- `name` (str) - Cluster name
- `description` (str) - Cluster description
- `keywords` (list[str]) - List of keyword strings
#### `save_output(parsed, original_data, account, progress_tracker, step_tracker)`
- **Input:**
- `parsed` - List of cluster dicts from parse_response
- `original_data` - Dict from prepare() with `keywords` and `sector_id`
- **Process:**
- Gets account, site, sector from first keyword
- For each cluster in parsed:
- Gets or creates `Clusters` record:
- Fields: `name`, `description`, `account`, `site`, `sector`, `status='active'`
- Uses `get_or_create` with name + account + site + sector
- Matches keywords (case-insensitive):
- Normalizes cluster keywords and available keywords to lowercase
- Updates matched `Keywords` records:
- Sets `cluster` foreign key
- Sets `status='mapped'`
- Recalculates cluster metrics:
- `keywords_count` - Count of keywords in cluster
- `volume` - Sum of keyword volumes (uses `volume_override` if available, else `seed_keyword__volume`)
- **Returns:** Dict with:
- `count` (int) - Clusters created
- `clusters_created` (int) - Clusters created
- `keywords_updated` (int) - Keywords updated
### 2.4 Database Models
#### Keywords Model
- **File:** `backend/igny8_core/modules/planner/models.py`
- **Model:** `Keywords`
- **Fields Used:**
- `id` - Keyword ID
- `seed_keyword` (ForeignKey) - Reference to SeedKeyword
- `keyword` (property) - Gets keyword text from seed_keyword
- `volume` (property) - Gets volume from volume_override or seed_keyword
- `difficulty` (property) - Gets difficulty from difficulty_override or seed_keyword
- `intent` (property) - Gets intent from seed_keyword
- `cluster` (ForeignKey) - Assigned cluster
- `status` - Status ('active', 'pending', 'mapped', 'archived')
- `account`, `site`, `sector` - From SiteSectorBaseModel
#### Clusters Model
- **File:** `backend/igny8_core/modules/planner/models.py`
- **Model:** `Clusters`
- **Fields Used:**
- `name` - Cluster name (unique)
- `description` - Cluster description
- `keywords_count` - Count of keywords (recalculated)
- `volume` - Sum of keyword volumes (recalculated)
- `status` - Status ('active')
- `account`, `site`, `sector` - From SiteSectorBaseModel
### 2.5 AI Response Format
**Expected JSON:**
```json
{
"clusters": [
{
"name": "Cluster Name",
"description": "Cluster description",
"keywords": ["keyword1", "keyword2", "keyword3"]
}
]
}
```
### 2.6 Progress Messages
- **INIT:** "Validating {keyword1}, {keyword2}, {keyword3} and {X} more keywords" (shows first 3, then count)
- **PREP:** "Loading {count} keyword(s)"
- **AI_CALL:** "Generating clusters with Igny8 Semantic SEO Model"
- **PARSE:** "{count} cluster(s) created"
- **SAVE:** "Saving {count} cluster(s)"
---
## 3. Generate Ideas
### 3.1 Function Implementation
- **File:** `backend/igny8_core/ai/functions/generate_ideas.py`
- **Class:** `GenerateIdeasFunction`
- **Inherits:** `BaseAIFunction`
### 3.2 API Endpoint
- **File:** `backend/igny8_core/modules/planner/views.py`
- **ViewSet:** `ClusterViewSet`
- **Action:** `auto_generate_ideas`
- **Method:** POST
- **URL Path:** `/v1/planner/clusters/auto_generate_ideas/`
- **Payload:**
- `ids` (list[int]) - Cluster IDs (max 10)
- **Response:**
- `success` (bool)
- `task_id` (str) - Celery task ID if async
- `ideas_created` (int) - Number of ideas created
- `message` (str)
### 3.3 Function Methods
#### `get_name()`
- **Returns:** `'generate_ideas'`
#### `get_metadata()`
- **Returns:** Dict with `display_name`, `description`, `phases` (INIT, PREP, AI_CALL, PARSE, SAVE, DONE)
#### `get_max_items()`
- **Returns:** `10` (max clusters per generation)
#### `validate(payload, account)`
- **Validates:**
- Calls `super().validate()` to check for 'ids' array and max_items limit
- Calls `validate_cluster_exists` for first cluster ID
- Calls `validate_cluster_limits` for plan limits (currently disabled)
- **Returns:** Dict with `valid` (bool) and optional `error` (str)
#### `prepare(payload, account)`
- **Loads:**
- Clusters from database (filters by `ids`, `account`)
- Uses `select_related` for: `sector`, `account`, `site`, `sector__site`
- Uses `prefetch_related` for: `keywords`
- **Gets Keywords:**
- For each cluster, loads `Keywords` with `select_related('seed_keyword')`
- Extracts keyword text from `seed_keyword.keyword`
- **Returns:** Dict with:
- `clusters` (list[Cluster objects])
- `cluster_data` (list[dict]) - Formatted data with: `id`, `name`, `description`, `keywords` (list[str])
- `account` (Account object)
#### `build_prompt(data, account)`
- **Gets Prompt:**
- Calls `PromptRegistry.get_prompt(function_name='generate_ideas', account, context)`
- Context includes:
- `CLUSTERS` - Formatted cluster list: `"Cluster ID: {id} | Name: {name} | Description: {description}"`
- `CLUSTER_KEYWORDS` - Formatted cluster keywords: `"Cluster ID: {id} | Name: {name} | Keywords: {keyword1}, {keyword2}"`
- **Replaces Placeholders:**
- `[IGNY8_CLUSTERS]` → clusters_text
- `[IGNY8_CLUSTER_KEYWORDS]` → cluster_keywords_text
- **Returns:** Prompt string
#### `parse_response(response, step_tracker)`
- **Parsing:**
- Calls `AICore.extract_json()` to extract JSON from response
- Validates 'ideas' key exists in JSON
- **Returns:** List[Dict] with idea data:
- `title` (str) - Idea title
- `description` (str or dict) - Idea description (can be JSON string)
- `content_type` (str) - Content type ('blog_post', 'article', etc.)
- `content_structure` (str) - Content structure ('cluster_hub', 'supporting_page', etc.)
- `cluster_id` (int, optional) - Cluster ID reference
- `cluster_name` (str, optional) - Cluster name reference
- `estimated_word_count` (int) - Estimated word count
- `covered_keywords` or `target_keywords` (str) - Target keywords
#### `save_output(parsed, original_data, account, progress_tracker, step_tracker)`
- **Input:**
- `parsed` - List of idea dicts from parse_response
- `original_data` - Dict from prepare() with `clusters` and `cluster_data`
- **Process:**
- For each idea in parsed:
- Matches cluster:
- First tries by `cluster_id` from AI response
- Falls back to `cluster_name` matching
- Last resort: position-based matching (first idea → first cluster)
- Gets site from cluster (or cluster.sector.site)
- Handles description:
- If dict, converts to JSON string
- If not string, converts to string
- Creates `ContentIdeas` record:
- Fields:
- `idea_title` - From `title`
- `description` - Processed description
- `content_type` - From `content_type` (default: 'blog_post')
- `content_structure` - From `content_structure` (default: 'supporting_page')
- `target_keywords` - From `covered_keywords` or `target_keywords`
- `keyword_cluster` - Matched cluster
- `estimated_word_count` - From `estimated_word_count` (default: 1500)
- `status` - 'new'
- `account`, `site`, `sector` - From cluster
- **Returns:** Dict with:
- `count` (int) - Ideas created
- `ideas_created` (int) - Ideas created
### 3.4 Database Models
#### Clusters Model
- **File:** `backend/igny8_core/modules/planner/models.py`
- **Model:** `Clusters`
- **Fields Used:**
- `id` - Cluster ID
- `name` - Cluster name
- `description` - Cluster description
- `keywords` (related_name) - Related Keywords
- `account`, `site`, `sector` - From SiteSectorBaseModel
#### ContentIdeas Model
- **File:** `backend/igny8_core/modules/planner/models.py`
- **Model:** `ContentIdeas`
- **Fields Used:**
- `idea_title` - Idea title
- `description` - Idea description (can be JSON string)
- `content_type` - Content type ('blog_post', 'article', 'guide', 'tutorial')
- `content_structure` - Content structure ('cluster_hub', 'landing_page', 'pillar_page', 'supporting_page')
- `target_keywords` - Target keywords string
- `keyword_cluster` (ForeignKey) - Related cluster
- `estimated_word_count` - Estimated word count
- `status` - Status ('new', 'scheduled', 'published')
- `account`, `site`, `sector` - From SiteSectorBaseModel
### 3.5 AI Response Format
**Expected JSON:**
```json
{
"ideas": [
{
"title": "Idea Title",
"description": "Idea description or JSON structure",
"content_type": "blog_post",
"content_structure": "supporting_page",
"cluster_id": 1,
"cluster_name": "Cluster Name",
"estimated_word_count": 1500,
"covered_keywords": "keyword1, keyword2"
}
]
}
```
### 3.6 Progress Messages
- **INIT:** "Verifying cluster integrity"
- **PREP:** "Loading cluster keywords"
- **AI_CALL:** "Generating ideas with Igny8 Semantic AI"
- **PARSE:** "{count} high-opportunity idea(s) generated"
- **SAVE:** "Content Outline for Ideas generated"
---
## 4. Generate Content
### 4.1 Function Implementation
- **File:** `backend/igny8_core/ai/functions/generate_content.py`
- **Class:** `GenerateContentFunction`
- **Inherits:** `BaseAIFunction`
### 4.2 API Endpoint
- **File:** `backend/igny8_core/modules/writer/views.py`
- **ViewSet:** `TasksViewSet`
- **Action:** `auto_generate_content`
- **Method:** POST
- **URL Path:** `/v1/writer/tasks/auto_generate_content/`
- **Payload:**
- `ids` (list[int]) - Task IDs (max 10)
- **Response:**
- `success` (bool)
- `task_id` (str) - Celery task ID if async
- `tasks_updated` (int) - Number of tasks updated
- `message` (str)
### 4.3 Function Methods
#### `get_name()`
- **Returns:** `'generate_content'`
#### `get_metadata()`
- **Returns:** Dict with `display_name`, `description`, `phases` (INIT, PREP, AI_CALL, PARSE, SAVE, DONE)
#### `get_max_items()`
- **Returns:** `50` (max tasks per batch)
#### `validate(payload, account)`
- **Validates:**
- Calls `super().validate()` to check for 'ids' array and max_items limit
- Calls `validate_tasks_exist` to verify tasks exist
- **Returns:** Dict with `valid` (bool) and optional `error` (str)
#### `prepare(payload, account)`
- **Loads:**
- Tasks from database (filters by `ids`, `account`)
- Uses `select_related` for: `account`, `site`, `sector`, `cluster`, `idea`
- **Returns:** List[Task objects]
#### `build_prompt(data, account)`
- **Input:** Can be single Task or list[Task] (handles first task if list)
- **Builds Idea Data:**
- `title` - From task.title
- `description` - From task.description
- `outline` - From task.idea.description (handles JSON structure):
- If JSON, formats as: `"## {H2 heading}\n### {H3 subheading}\nContent Type: {type}\nDetails: {details}"`
- If plain text, uses as-is
- `structure` - From task.idea.content_structure or task.content_structure
- `type` - From task.idea.content_type or task.content_type
- `estimated_word_count` - From task.idea.estimated_word_count
- **Builds Cluster Data:**
- `cluster_name` - From task.cluster.name
- `description` - From task.cluster.description
- `status` - From task.cluster.status
- **Builds Keywords Data:**
- From task.keywords (legacy) or task.idea.target_keywords
- **Gets Prompt:**
- Calls `PromptRegistry.get_prompt(function_name='generate_content', account, task, context)`
- Context includes:
- `IDEA` - Formatted idea data string
- `CLUSTER` - Formatted cluster data string
- `KEYWORDS` - Keywords string
- **Returns:** Prompt string
#### `parse_response(response, step_tracker)`
- **Parsing:**
- First tries JSON parse:
- If successful and dict, returns dict
- Falls back to plain text:
- Calls `normalize_content()` from `content_normalizer` to convert to HTML
- Returns dict with `content` field
- **Returns:** Dict with:
- **If JSON:**
- `content` (str) - HTML content
- `title` (str, optional) - Content title
- `meta_title` (str, optional) - Meta title
- `meta_description` (str, optional) - Meta description
- `word_count` (int, optional) - Word count
- `primary_keyword` (str, optional) - Primary keyword
- `secondary_keywords` (list, optional) - Secondary keywords
- `tags` (list, optional) - Tags
- `categories` (list, optional) - Categories
- **If Plain Text:**
- `content` (str) - Normalized HTML content
#### `save_output(parsed, original_data, account, progress_tracker, step_tracker)`
- **Input:**
- `parsed` - Dict from parse_response
- `original_data` - Task object or list[Task] (handles first task if list)
- **Process:**
- Extracts content fields from parsed dict:
- `content_html` - From `content` field
- `title` - From `title` or task.title
- `meta_title` - From `meta_title` or task.meta_title or task.title
- `meta_description` - From `meta_description` or task.meta_description or task.description
- `word_count` - From `word_count` or calculated from content
- `primary_keyword` - From `primary_keyword`
- `secondary_keywords` - From `secondary_keywords` (converts to list if needed)
- `tags` - From `tags` (converts to list if needed)
- `categories` - From `categories` (converts to list if needed)
- Calculates word count if not provided:
- Strips HTML tags and counts words
- Gets or creates `Content` record:
- Uses `get_or_create` with `task` (OneToOne relationship)
- Defaults: `html_content`, `word_count`, `status='draft'`, `account`, `site`, `sector`
- Updates `Content` fields:
- `html_content` - Content HTML
- `word_count` - Word count
- `title` - Content title
- `meta_title` - Meta title
- `meta_description` - Meta description
- `primary_keyword` - Primary keyword
- `secondary_keywords` - Secondary keywords (JSONField)
- `tags` - Tags (JSONField)
- `categories` - Categories (JSONField)
- `status` - Always 'draft' for newly generated content
- `metadata` - Extra fields from parsed dict (excludes standard fields)
- `account`, `site`, `sector`, `task` - Aligned from task
- Updates `Tasks` record:
- Sets `status='completed'`
- Updates `updated_at`
- **Returns:** Dict with:
- `count` (int) - Tasks updated (always 1 per task)
- `tasks_updated` (int) - Tasks updated
- `word_count` (int) - Word count
### 4.4 Database Models
#### Tasks Model
- **File:** `backend/igny8_core/modules/writer/models.py`
- **Model:** `Tasks`
- **Fields Used:**
- `id` - Task ID
- `title` - Task title
- `description` - Task description
- `keywords` - Keywords string (legacy)
- `cluster` (ForeignKey) - Related cluster
- `idea` (ForeignKey) - Related ContentIdeas
- `content_structure` - Content structure
- `content_type` - Content type
- `status` - Status ('queued', 'completed')
- `meta_title` - Meta title
- `meta_description` - Meta description
- `account`, `site`, `sector` - From SiteSectorBaseModel
#### Content Model
- **File:** `backend/igny8_core/modules/writer/models.py`
- **Model:** `Content`
- **Fields Used:**
- `task` (OneToOneField) - Related task
- `html_content` - HTML content
- `word_count` - Word count
- `title` - Content title
- `meta_title` - Meta title
- `meta_description` - Meta description
- `primary_keyword` - Primary keyword
- `secondary_keywords` (JSONField) - Secondary keywords list
- `tags` (JSONField) - Tags list
- `categories` (JSONField) - Categories list
- `status` - Status ('draft', 'review', 'published')
- `metadata` (JSONField) - Additional metadata
- `account`, `site`, `sector` - From SiteSectorBaseModel (auto-set from task)
### 4.5 AI Response Format
**Expected JSON:**
```json
{
"content": "<html>Content HTML</html>",
"title": "Content Title",
"meta_title": "Meta Title",
"meta_description": "Meta description",
"word_count": 1500,
"primary_keyword": "primary keyword",
"secondary_keywords": ["keyword1", "keyword2"],
"tags": ["tag1", "tag2"],
"categories": ["category1"]
}
```
**Or Plain Text:**
```
Plain text content that will be normalized to HTML
```
### 4.6 Progress Messages
- **INIT:** "Validating task"
- **PREP:** "Preparing content idea"
- **AI_CALL:** "Writing article with Igny8 Semantic AI"
- **PARSE:** "{count} article(s) created"
- **SAVE:** "Saving article"
---
## 5. Change Guide
### 5.1 Where to Change Validation Logic
- **File:** `backend/igny8_core/ai/validators.py`
- **Functions:** `validate_ids`, `validate_keywords_exist`, `validate_cluster_exists`, `validate_tasks_exist`
- **Or:** Override `validate()` method in function class
### 5.2 Where to Change Data Loading
- **File:** Function-specific file (e.g., `auto_cluster.py`)
- **Method:** `prepare()`
- **Change:** Modify queryset filters, select_related, prefetch_related
### 5.3 Where to Change Prompts
- **File:** `backend/igny8_core/ai/prompts.py`
- **Method:** `PromptRegistry.get_prompt()`
- **Change:** Modify `DEFAULT_PROMPTS` dict or update database prompts
### 5.4 Where to Change Model Configuration
- **File:** `backend/igny8_core/ai/settings.py`
- **Constant:** `MODEL_CONFIG`
- **Change:** Update model, max_tokens, temperature, response_format per function
### 5.5 Where to Change Response Parsing
- **File:** Function-specific file (e.g., `generate_content.py`)
- **Method:** `parse_response()`
- **Change:** Modify JSON extraction or plain text handling
### 5.6 Where to Change Database Saving
- **File:** Function-specific file (e.g., `auto_cluster.py`)
- **Method:** `save_output()`
- **Change:** Modify model creation/update logic, field mappings
### 5.7 Where to Change Progress Messages
- **File:** `backend/igny8_core/ai/engine.py`
- **Methods:** `_get_prep_message()`, `_get_ai_call_message()`, `_get_parse_message()`, `_get_save_message()`
- **Or:** Override in function class `get_metadata()` phases
### 5.8 Where to Change Error Handling
- **File:** `backend/igny8_core/ai/engine.py`
- **Method:** `_handle_error()`
- **Change:** Modify error logging, error response format
---
## 6. Dependencies
### 6.1 Function Dependencies
- All functions depend on: `BaseAIFunction`, `AICore`, `PromptRegistry`, `get_model_config`
- Clustering depends on: `Keywords`, `Clusters` models
- Ideas depends on: `Clusters`, `ContentIdeas`, `Keywords` models
- Content depends on: `Tasks`, `Content`, `ContentIdeas`, `Clusters` models
### 6.2 External Dependencies
- **Celery:** For async task execution (`run_ai_task`)
- **OpenAI API:** For AI text generation (via `AICore.run_ai_request`)
- **Django ORM:** For database operations
- **IntegrationSettings:** For account-specific model configuration
---
## 7. Key Relationships
### 7.1 Clustering Flow
```
Keywords → Clusters (many-to-one)
- Keywords.cluster (ForeignKey)
- Clusters.keywords (related_name)
```
### 7.2 Ideas Flow
```
Clusters → ContentIdeas (one-to-many)
- ContentIdeas.keyword_cluster (ForeignKey)
- Clusters.ideas (related_name, if exists)
```
### 7.3 Content Flow
```
Tasks → Content (one-to-one)
- Content.task (OneToOneField)
- Tasks.content_record (related_name)
Tasks → ContentIdeas (many-to-one)
- Tasks.idea (ForeignKey)
- ContentIdeas.tasks (related_name)
Tasks → Clusters (many-to-one)
- Tasks.cluster (ForeignKey)
- Clusters.tasks (related_name)
```
---
## 8. Notes
- All functions use the same execution pipeline through `AIEngine.execute()`
- Progress tracking is handled automatically by `AIEngine`
- Cost tracking is handled automatically by `CostTracker`
- Database logging is handled automatically by `AITaskLog`
- Model configuration can be overridden per account via `IntegrationSettings`
- Prompts can be overridden per account via database prompts
- All functions support both async (Celery) and sync execution
- Error handling is centralized in `AIEngine._handle_error()`