# IGNY8 Backend Documentation

**Last Updated:** 2025-01-XX  
**Purpose:** Complete backend documentation covering models, views, APIs, modules, serializers, tasks, and structure.

---

## Table of Contents

1. [Backend Overview](#backend-overview)
2. [Tech Stack](#tech-stack)
3. [Project Structure](#project-structure)
4. [Models](#models)
5. [ViewSets](#viewsets)
6. [Serializers](#serializers)
7. [Celery Tasks](#celery-tasks)
8. [API Endpoints](#api-endpoints)
9. [Base Classes](#base-classes)
10. [Middleware](#middleware)
11. [Utilities](#utilities)
12. [Modules](#modules)

---

## Backend Overview

The IGNY8 backend is a Django 5.2+ application using Django REST Framework (DRF) for API endpoints. The backend follows a modular architecture with clear separation of concerns, automatic account isolation, and support for asynchronous task processing via Celery.

### Key Features

- **Multi-Tenancy**: Complete account isolation with automatic filtering
- **RESTful API**: DRF ViewSets with consistent response format
- **Celery Integration**: Asynchronous task processing for long-running operations
- **Account/Site/Sector Hierarchy**: Hierarchical data organization
- **AI Integration**: Unified AI framework for all AI operations
- **Progress Tracking**: Real-time progress updates for Celery tasks

---

## Tech Stack

### Core Technologies

- **Django 5.2+**: Web framework
- **Django REST Framework**: API framework
- **PostgreSQL**: Database
- **Celery**: Asynchronous task queue
- **Redis**: Celery broker and caching

### Key Libraries

- **django-filter**: Advanced filtering
- **djangorestframework-simplejwt**: JWT authentication
- **requests**: HTTP client for external APIs
- **python-dotenv**: Environment variable management

---

## Project Structure

```
backend/igny8_core/
├── auth/                     # Multi-tenancy and authentication
│   ├── models.py             # Account, User, Plan, Site, Sector, Industry models
│   ├── views.py              # Account, User, Site, Sector ViewSets
│   ├── serializers.py        # Account, User, Plan serializers
│   └── urls.py               # Auth module URLs
├── modules/                  # Feature modules
│   ├── planner/              # Keywords, Clusters, Ideas
│   │   ├── models.py         # Keywords, Clusters, ContentIdeas models
│   │   ├── views.py          # KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
│   │   ├── tasks.py          # Celery tasks for AI operations
│   │   ├── serializers.py    # Model serializers
│   │   └── urls.py           # Planner module URLs
│   ├── writer/               # Tasks, Content, Images
│   │   ├── models.py         # Tasks, Content, Images models
│   │   ├── views.py          # TasksViewSet, ImagesViewSet
│   │   ├── tasks.py          # Celery tasks for content/image generation
│   │   └── urls.py           # Writer module URLs
│   ├── system/               # Settings, Prompts, Integration
│   │   ├── models.py         # AIPrompt, IntegrationSettings, AuthorProfile, Strategy
│   │   ├── views.py          # AIPromptViewSet, AuthorProfileViewSet
│   │   ├── integration_views.py  # IntegrationSettingsViewSet, task_progress
│   │   ├── utils.py          # Default prompts, prompt loading
│   │   └── urls.py           # System module URLs
│   └── billing/              # Credits, Transactions, Usage
│       ├── models.py         # CreditTransaction, CreditUsageLog models
│       ├── views.py          # Billing ViewSets
│       └── services.py       # CreditService
├── api/                      # API base classes
│   ├── base.py               # AccountModelViewSet, SiteSectorModelViewSet
│   └── pagination.py         # CustomPageNumberPagination
├── utils/                    # Shared utilities
│   ├── ai_processor.py       # Unified AI interface (legacy, see AI functions)
│   └── content_normalizer.py # Content processing utilities
├── middleware/               # Custom middleware
│   ├── account.py            # AccountContextMiddleware (sets request.account)
│   └── resource_tracker.py   # ResourceTrackerMiddleware (API metrics)
├── ai/                       # AI framework
│   ├── base.py               # BaseAIFunction
│   ├── engine.py             # AIEngine
│   ├── tasks.py              # run_ai_task
│   ├── registry.py           # Function registry
│   ├── prompts.py            # PromptRegistry
│   ├── ai_core.py            # AICore
│   ├── settings.py           # Model configuration
│   ├── validators.py         # Validation functions
│   ├── tracker.py            # Progress tracking
│   └── functions/            # AI function implementations
│       ├── auto_cluster.py
│       ├── generate_ideas.py
│       ├── generate_content.py
│       ├── generate_image_prompts.py
│       └── generate_images.py
├── settings.py               # Django settings
├── urls.py                   # Root URL configuration
└── celery.py                 # Celery configuration
```

---

## Models

### Base Models

#### AccountBaseModel
**File**: `auth/models.py`

**Purpose**: Base model for all account-isolated models.

**Fields**:
- `account`: ForeignKey to Account
- `created_at`: DateTimeField (auto_now_add)
- `updated_at`: DateTimeField (auto_now)

**Usage**: All models that need account isolation inherit from this.

#### SiteSectorBaseModel
**File**: `auth/models.py`

**Purpose**: Base model for models that belong to Site and Sector.

**Fields**:
- Inherits from `AccountBaseModel`
- `site`: ForeignKey to Site
- `sector`: ForeignKey to Sector

**Methods**:
- `save()`: Automatically sets `account` from `site.account` and validates sector belongs to site

**Usage**: Models like Keywords, Clusters, ContentIdeas, Tasks inherit from this.

### Auth Models

#### Account
**Table**: `igny8_accounts`

**Fields**:
- `name`: CharField
- `slug`: SlugField (unique)
- `owner`: ForeignKey to User
- `plan`: ForeignKey to Plan
- `credits`: IntegerField (default: 0)
- `status`: CharField (choices: active, suspended, trial, cancelled)

**Methods**:
- `is_system_account()`: Returns True if account is a system account

#### User
**Table**: `igny8_users`

**Inherits**: `AbstractUser`

**Fields**:
- `email`: EmailField (unique)
- `account`: ForeignKey to Account
- `role`: CharField (choices: developer, owner, admin, editor, viewer, system_bot)

**Methods**:
- `has_role(role)`: Checks if user has role
- `is_owner_or_admin()`: Checks if user is owner or admin
- `is_developer()`: Checks if user is developer
- `is_admin_or_developer()`: Checks if user is admin or developer
- `is_system_account_user()`: Checks if user belongs to system account
- `get_accessible_sites()`: Returns queryset of accessible sites

#### Plan
**Table**: `igny8_plans`

**Fields**: Extensive fields for limits (users, sites, keywords, clusters, content ideas, AI requests, word count, images, credits)

**Methods**:
- `clean()`: Validates plan limits
- `get_effective_credits_per_month()`: Returns included_credits or credits_per_month

#### Site
**Table**: `igny8_sites`

**Inherits**: `AccountBaseModel`

**Fields**:
- `name`: CharField
- `slug`: SlugField (unique per account)
- `domain`: URLField (optional)
- `industry`: ForeignKey to Industry (optional)
- `is_active`: BooleanField
- `status`: CharField
- `wp_url`: URLField (optional)
- `wp_username`: CharField (optional)
- `wp_app_password`: CharField (optional)

**Methods**:
- `get_active_sectors_count()`: Returns count of active sectors
- `can_add_sector()`: Returns True if site can add another sector (max 5)

#### Sector
**Table**: `igny8_sectors`

**Inherits**: `AccountBaseModel`

**Fields**:
- `site`: ForeignKey to Site
- `industry_sector`: ForeignKey to IndustrySector (optional)
- `name`: CharField
- `slug`: SlugField (unique per site)
- `is_active`: BooleanField
- `status`: CharField

**Validation**: Maximum 5 active sectors per site

### Planner Models

#### Keywords
**Table**: `igny8_keywords`

**Inherits**: `SiteSectorBaseModel`

**Fields**:
- `keyword`: CharField
- `volume`: IntegerField (optional)
- `difficulty`: IntegerField (optional)
- `intent`: CharField (optional)
- `status`: CharField
- `cluster`: ManyToManyField to Clusters

#### Clusters
**Table**: `igny8_clusters`

**Inherits**: `SiteSectorBaseModel`

**Fields**:
- `name`: CharField
- `description`: TextField (optional)
- `keywords_count`: IntegerField (calculated)
- `volume`: IntegerField (calculated)
- `status`: CharField
- `keywords`: ManyToManyField to Keywords

#### ContentIdeas
**Table**: `igny8_content_ideas`

**Inherits**: `SiteSectorBaseModel`

**Fields**:
- `idea_title`: CharField
- `description`: TextField
- `content_type`: CharField
- `content_structure`: CharField
- `target_keywords`: TextField
- `keyword_cluster`: ForeignKey to Clusters
- `estimated_word_count`: IntegerField
- `status`: CharField

### Writer Models

#### Tasks
**Table**: `igny8_tasks`

**Inherits**: `SiteSectorBaseModel`

**Fields**:
- `title`: CharField
- `description`: TextField
- `cluster`: ForeignKey to Clusters (optional)
- `idea`: ForeignKey to ContentIdeas (optional)
- `content_type`: CharField
- `content_structure`: CharField
- `status`: CharField

#### Content
**Table**: `igny8_content`

**Inherits**: `SiteSectorBaseModel`

**Fields**:
- `task`: OneToOneField to Tasks
- `html_content`: TextField
- `word_count`: IntegerField
- `status`: CharField
- `wp_post_id`: IntegerField (optional)
- `meta_title`: CharField (optional)
- `meta_description`: TextField (optional)
- `primary_keyword`: CharField (optional)
- `secondary_keywords`: TextField (optional)

#### Images
**Table**: `igny8_images`

**Inherits**: `SiteSectorBaseModel`

**Fields**:
- `task`: ForeignKey to Tasks (optional)
- `content`: ForeignKey to Content (optional)
- `image_type`: CharField (choices: featured, in_article, desktop, mobile)
- `prompt`: TextField
- `image_url`: CharField
- `status`: CharField

### System Models

#### AIPrompt
**Table**: `igny8_ai_prompts`

**Inherits**: `AccountBaseModel`

**Fields**:
- `prompt_type`: CharField
- `prompt_value`: TextField
- `function_name`: CharField (optional)

#### IntegrationSettings
**Table**: `igny8_integration_settings`

**Inherits**: `AccountBaseModel`

**Fields**:
- `integration_type`: CharField (choices: openai, runware, image_generation)
- `config`: JSONField

#### AuthorProfile
**Table**: `igny8_author_profiles`

**Inherits**: `AccountBaseModel`

**Fields**:
- `name`: CharField
- `description`: TextField
- `tone`: CharField
- `language`: CharField

#### Strategy
**Table**: `igny8_strategies`

**Inherits**: `AccountBaseModel`

**Fields**:
- `name`: CharField
- `description`: TextField
- `sector`: ForeignKey to Sector (optional)
- `prompt_types`: JSONField
- `section_logic`: JSONField
- `is_active`: BooleanField

### Billing Models

#### CreditTransaction
**Table**: `igny8_credit_transactions`

**Inherits**: `AccountBaseModel`

**Fields**:
- `transaction_type`: CharField (choices: purchase, usage, refund, adjustment)
- `amount`: IntegerField
- `balance_after`: IntegerField
- `description`: TextField (optional)

#### CreditUsageLog
**Table**: `igny8_credit_usage_logs`

**Inherits**: `AccountBaseModel`

**Fields**:
- `operation_type`: CharField
- `credits_used`: IntegerField
- `cost_usd`: DecimalField
- `details`: JSONField (optional)

---

## ViewSets

### Base ViewSets

#### AccountModelViewSet
**File**: `api/base.py`

**Purpose**: Base ViewSet with automatic account filtering.

**Methods**:
- `get_queryset()`: Filters queryset by `request.account` (with admin/developer override)
- `perform_create()`: Sets account on created objects
- `get_serializer_context()`: Adds account to serializer context

**Access Control**:
- Admin/Developer users: Bypass account filtering
- System account users: Bypass account filtering
- Regular users: Only see data from their account

#### SiteSectorModelViewSet
**File**: `api/base.py`

**Purpose**: Base ViewSet with site/sector filtering and access control.

**Inherits**: `AccountModelViewSet`

**Methods**:
- `get_queryset()`: Filters by account, accessible sites, and optional site_id/sector_id
- `perform_create()`: Validates site access and sector-site relationship
- `get_serializer_context()`: Adds accessible sites and sectors to context

**Access Control**:
- Developers: All active sites
- System account users: All active sites
- Owners/Admins: All sites in their account
- Editors/Viewers: Only sites granted via `SiteUserAccess`

### Planner ViewSets

#### KeywordViewSet
**Inherits**: `SiteSectorModelViewSet`

**Actions**:
- `list()`: List keywords with filtering
- `create()`: Create keyword
- `retrieve()`: Get keyword details
- `update()`: Update keyword
- `destroy()`: Delete keyword
- `auto_cluster()`: Auto-cluster keywords using AI
- `bulk_delete()`: Bulk delete keywords
- `bulk_update_status()`: Bulk update keyword status
- `export_csv()`: Export keywords to CSV
- `import_csv()`: Import keywords from CSV

**Filtering**:
- Search: `keyword` field
- Filters: `status`, `cluster_id`, `intent`
- Custom: `difficulty_min`, `difficulty_max`, `volume_min`, `volume_max`
- Ordering: `created_at`, `volume`, `difficulty`

#### ClusterViewSet
**Inherits**: `SiteSectorModelViewSet`

**Actions**:
- `list()`: List clusters
- `create()`: Create cluster
- `retrieve()`: Get cluster details
- `update()`: Update cluster
- `destroy()`: Delete cluster
- `auto_generate_ideas()`: Auto-generate content ideas for clusters

#### ContentIdeasViewSet
**Inherits**: `SiteSectorModelViewSet`

**Actions**:
- `list()`: List content ideas
- `create()`: Create content idea
- `retrieve()`: Get content idea details
- `update()`: Update content idea
- `destroy()`: Delete content idea

### Writer ViewSets

#### TasksViewSet
**Inherits**: `SiteSectorModelViewSet`

**Actions**:
- `list()`: List tasks
- `create()`: Create task
- `retrieve()`: Get task details
- `update()`: Update task
- `destroy()`: Delete task
- `auto_generate_content()`: Auto-generate content for tasks
- `generate_images()`: Generate images for image records
- `bulk_delete()`: Bulk delete tasks
- `bulk_update()`: Bulk update task status

**Filtering**:
- Search: `title`, `keywords`
- Filters: `status`, `cluster_id`, `content_type`, `content_structure`
- Ordering: `title`, `created_at`, `word_count`, `status`

#### ImagesViewSet
**Inherits**: `SiteSectorModelViewSet`

**Actions**:
- `list()`: List images
- `create()`: Create image
- `retrieve()`: Get image details
- `update()`: Update image
- `destroy()`: Delete image
- `generate_images()`: Generate images using AI

### System ViewSets

#### IntegrationSettingsViewSet
**Inherits**: `viewsets.ViewSet`

**Actions**:
- `list()`: List integrations
- `retrieve()`: Get integration settings
- `update()`: Save integration settings
- `save_post()`: Save integration settings (POST)
- `test_connection()`: Test API connection
- `test_openai()`: Test OpenAI connection
- `test_runware()`: Test Runware connection
- `generate_image()`: Test image generation
- `task_progress()`: Get Celery task progress

#### AIPromptViewSet
**Inherits**: `AccountModelViewSet`

**Actions**:
- `list()`: List prompts
- `create()`: Create prompt
- `retrieve()`: Get prompt details
- `update()`: Update prompt
- `destroy()`: Delete prompt
- `reset_to_default()`: Reset prompt to default value

#### AuthorProfileViewSet
**Inherits**: `AccountModelViewSet`

**Actions**:
- `list()`: List author profiles
- `create()`: Create author profile
- `retrieve()`: Get author profile details
- `update()`: Update author profile
- `destroy()`: Delete author profile

---

## Serializers

### Planner Serializers

#### KeywordSerializer
**Fields**: All Keyword model fields

**Validation**: Validates keyword uniqueness, cluster belongs to same sector

#### ClusterSerializer
**Fields**: All Cluster model fields

**Read-Only Fields**: `keywords_count`, `volume` (calculated)

#### ContentIdeasSerializer
**Fields**: All ContentIdeas model fields

### Writer Serializers

#### TasksSerializer
**Fields**: All Tasks model fields

#### ContentSerializer
**Fields**: All Content model fields

#### ImagesSerializer
**Fields**: All Images model fields

### System Serializers

#### AIPromptSerializer
**Fields**: All AIPrompt model fields

#### IntegrationSettingsSerializer
**Fields**: All IntegrationSettings model fields

---

## Celery Tasks

### AI Task Entry Point

#### run_ai_task
**File**: `ai/tasks.py`

**Purpose**: Unified Celery task entrypoint for all AI functions

**Parameters**:
- `function_name`: Function name (e.g., 'auto_cluster')
- `payload`: Function-specific payload
- `account_id`: Account ID

**Flow**:
1. Gets account from account_id
2. Gets function instance from registry
3. Creates AIEngine
4. Executes function via AIEngine

### Planner Tasks

#### auto_cluster_keywords_task
**File**: `modules/planner/tasks.py` (legacy, now uses `run_ai_task`)

**Purpose**: Auto-cluster keywords using AI

**Parameters**:
- `keyword_ids`: List of keyword IDs
- `account_id`: Account ID
- `site_id`: Site ID
- `sector_id`: Sector ID

**Progress Tracking**: Updates progress with request_steps and response_steps

#### auto_generate_ideas_task
**File**: `modules/planner/tasks.py` (legacy, now uses `run_ai_task`)

**Purpose**: Auto-generate content ideas for clusters

**Parameters**:
- `cluster_ids`: List of cluster IDs
- `account_id`: Account ID

**Progress Tracking**: Updates progress for each cluster

### Writer Tasks

#### auto_generate_content_task
**File**: `modules/writer/tasks.py` (legacy, now uses `run_ai_task`)

**Purpose**: Auto-generate content for tasks

**Parameters**:
- `task_ids`: List of task IDs
- `account_id`: Account ID

**Progress Tracking**: Updates progress for each task

#### process_image_generation_queue
**File**: `modules/writer/views.py` (via `ai/tasks.py`)

**Purpose**: Generate images for image records

**Parameters**:
- `image_ids`: List of image IDs
- `account_id`: Account ID
- `content_id`: Content ID (optional)

**Progress Tracking**: Updates progress for each image sequentially

---

## API Endpoints

### Base URL

`/api/v1/`

### Planner Endpoints

- `GET /api/v1/planner/keywords/` - List keywords
- `POST /api/v1/planner/keywords/` - Create keyword
- `GET /api/v1/planner/keywords/{id}/` - Get keyword
- `PUT /api/v1/planner/keywords/{id}/` - Update keyword
- `DELETE /api/v1/planner/keywords/{id}/` - Delete keyword
- `POST /api/v1/planner/keywords/auto_cluster/` - Auto-cluster keywords
- `POST /api/v1/planner/keywords/bulk_delete/` - Bulk delete keywords
- `POST /api/v1/planner/keywords/bulk_update_status/` - Bulk update status
- `GET /api/v1/planner/keywords/export_csv/` - Export keywords
- `POST /api/v1/planner/keywords/import_csv/` - Import keywords
- `GET /api/v1/planner/clusters/` - List clusters
- `POST /api/v1/planner/clusters/auto_generate_ideas/` - Auto-generate ideas
- `GET /api/v1/planner/ideas/` - List content ideas

### Writer Endpoints

- `GET /api/v1/writer/tasks/` - List tasks
- `POST /api/v1/writer/tasks/auto_generate_content/` - Auto-generate content
- `POST /api/v1/writer/images/generate_images/` - Generate images

### System Endpoints

- `GET /api/v1/system/settings/integrations/{pk}/` - Get integration settings
- `PUT /api/v1/system/settings/integrations/{pk}/` - Save integration settings
- `POST /api/v1/system/settings/integrations/{pk}/test_openai/` - Test OpenAI
- `POST /api/v1/system/settings/integrations/{pk}/test_runware/` - Test Runware
- `POST /api/v1/system/settings/integrations/{pk}/generate_image/` - Test image generation
- `GET /api/v1/system/settings/task_progress/{task_id}/` - Get task progress

---

## Base Classes

### AccountModelViewSet

**File**: `api/base.py`

**Purpose**: Base ViewSet with automatic account filtering

**Features**:
- Automatic account filtering
- Admin/Developer override
- Account context in serializers

### SiteSectorModelViewSet

**File**: `api/base.py`

**Purpose**: Base ViewSet with site/sector filtering

**Features**:
- Account filtering (inherited)
- Site access control
- Sector validation
- Accessible sites/sectors in serializer context

---

## Middleware

### AccountContextMiddleware

**File**: `middleware/account.py`

**Purpose**: Sets `request.account` from JWT token

**Functionality**:
- Extracts account ID from JWT token
- Loads Account object
- Sets `request.account`

### ResourceTrackerMiddleware

**File**: `middleware/resource_tracker.py`

**Purpose**: Tracks API request metrics

**Functionality**:
- Tracks CPU, memory, I/O usage
- Stores metrics in cache
- Provides metrics endpoint

---

## Utilities

### Content Normalizer

**File**: `utils/content_normalizer.py`

**Purpose**: Content processing utilities

**Functions**:
- `normalize_content()`: Converts plain text to HTML
- `_extract_body_content()`: Extracts body content from HTML

---

## Modules

### Planner Module

**Purpose**: Keyword management and content planning

**Models**: Keywords, Clusters, ContentIdeas

**ViewSets**: KeywordViewSet, ClusterViewSet, ContentIdeasViewSet

**Tasks**: Auto-cluster keywords, Auto-generate ideas

### Writer Module

**Purpose**: Content generation and management

**Models**: Tasks, Content, Images

**ViewSets**: TasksViewSet, ImagesViewSet

**Tasks**: Auto-generate content, Generate images

### System Module

**Purpose**: System configuration and AI settings

**Models**: AIPrompt, IntegrationSettings, AuthorProfile, Strategy

**ViewSets**: AIPromptViewSet, IntegrationSettingsViewSet, AuthorProfileViewSet

### Billing Module

**Purpose**: Credit management and usage tracking

**Models**: CreditTransaction, CreditUsageLog

**ViewSets**: CreditTransactionViewSet, CreditUsageLogViewSet

**Services**: CreditService (check, deduct, add, calculate credits)

---

## Summary

The IGNY8 backend provides:

1. **Multi-Tenancy**: Complete account isolation with automatic filtering
2. **RESTful API**: DRF ViewSets with consistent response format
3. **Celery Integration**: Asynchronous task processing
4. **Hierarchical Organization**: Account > Site > Sector > Content structure
5. **AI Framework**: Unified AI framework for all AI operations
6. **Progress Tracking**: Real-time progress updates for Celery tasks
7. **Module-Based Design**: Clear separation of concerns
8. **Base Classes**: Reusable ViewSets for common patterns
9. **Middleware**: Account context and resource tracking
10. **Utilities**: Content processing and AI integration

This architecture ensures scalability, maintainability, and extensibility while providing a robust foundation for the IGNY8 platform.

