# IGNY8 Backend Documentation

**Version:** 1.0  
**Last Updated:** 2025-01-XX  
**Purpose:** Complete backend documentation covering models, views, APIs, modules, serializers, tasks, and structure.

---

## Table of Contents

1. [Backend Overview](#backend-overview)
2. [Tech Stack](#tech-stack)
3. [Project Structure](#project-structure)
4. [Models](#models)
5. [ViewSets](#viewsets)
6. [Serializers](#serializers)
7. [Celery Tasks](#celery-tasks)
8. [API Endpoints](#api-endpoints)
9. [Base Classes](#base-classes)
10. [Middleware](#middleware)
11. [Utilities](#utilities)
12. [Modules](#modules)

---

## Backend Overview

The IGNY8 backend is a Django 5.2+ application using Django REST Framework (DRF) for API endpoints. The backend follows a modular architecture with clear separation of concerns, automatic account isolation, and support for asynchronous task processing via Celery.

### Key Features

- **Multi-Tenancy**: Complete account isolation with automatic filtering
- **RESTful API**: DRF ViewSets with consistent response format
- **Celery Integration**: Asynchronous task processing for long-running operations
- **Account/Site/Sector Hierarchy**: Hierarchical data organization
- **AI Integration**: Unified AIProcessor for all AI operations
- **Progress Tracking**: Real-time progress updates for Celery tasks

---

## Tech Stack

### Core Technologies

- **Django 5.2+**: Web framework
- **Django REST Framework**: API framework
- **PostgreSQL**: Database
- **Celery**: Asynchronous task queue
- **Redis**: Celery broker and caching

### Key Libraries

- **django-filter**: Advanced filtering
- **djangorestframework-simplejwt**: JWT authentication
- **requests**: HTTP client for external APIs
- **python-dotenv**: Environment variable management

---

## Project Structure

```
backend/igny8_core/
├── auth/                     # Multi-tenancy and authentication
│   ├── models.py             # Account, User, Plan, Site, Sector, Industry models
│   ├── views.py              # Account, User, Site, Sector ViewSets
│   ├── serializers.py        # Account, User, Plan serializers
│   └── urls.py               # Auth module URLs
├── modules/                  # Feature modules
│   ├── planner/              # Keywords, Clusters, Ideas
│   │   ├── models.py         # Keywords, Clusters, ContentIdeas models
│   │   ├── views.py          # KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
│   │   ├── tasks.py          # Celery tasks for AI operations
│   │   ├── serializers.py    # Model serializers
│   │   └── urls.py           # Planner module URLs
│   ├── writer/               # Tasks, Content, Images
│   │   ├── models.py         # Tasks, Content, Images models
│   │   ├── views.py          # TasksViewSet
│   │   ├── tasks.py          # Celery tasks for content/image generation
│   │   └── urls.py           # Writer module URLs
│   ├── system/               # Settings, Prompts, Integration
│   │   ├── models.py         # AIPrompt, IntegrationSettings, AuthorProfile, Strategy
│   │   ├── views.py          # AIPromptViewSet, AuthorProfileViewSet
│   │   ├── integration_views.py  # IntegrationSettingsViewSet, task_progress
│   │   ├── utils.py          # Default prompts, prompt loading
│   │   └── urls.py           # System module URLs
│   └── billing/              # Credits, Transactions, Usage
│       ├── models.py         # CreditTransaction, UsageLog models
│       ├── views.py          # Billing ViewSets
│       └── services.py       # CreditService
├── api/                      # API base classes
│   ├── base.py               # AccountModelViewSet, SiteSectorModelViewSet
│   └── pagination.py         # CustomPageNumberPagination
├── utils/                    # Shared utilities
│   ├── ai_processor.py       # Unified AI interface
│   └── content_normalizer.py # Content processing utilities
├── middleware/               # Custom middleware
│   ├── account.py            # AccountContextMiddleware (sets request.account)
│   └── resource_tracker.py   # ResourceTrackerMiddleware (API metrics)
├── settings.py               # Django settings
├── urls.py                   # Root URL configuration
└── celery.py                 # Celery configuration
```

---

## Models

### Base Models

#### AccountBaseModel
**File**: `auth/models.py`

**Purpose**: Base model for all account-isolated models.

**Fields**:
- `account`: ForeignKey to Account
- `created_at`: DateTimeField (auto_now_add)
- `updated_at`: DateTimeField (auto_now)

**Usage**: All models that need account isolation inherit from this.


#### SiteSectorBaseModel
**File**: `auth/models.py`

**Purpose**: Base model for models that belong to Site and Sector.

**Fields**:
- Inherits from `AccountBaseModel`
- `site`: ForeignKey to Site
- `sector`: ForeignKey to Sector

**Methods**:
- `save()`: Automatically sets `account` from `site.account` and validates sector belongs to site

**Usage**: Models like Keywords, Clusters, ContentIdeas, Tasks inherit from this.

### Auth Models

#### Account
**File**: `auth/models.py`

**Table**: `igny8_accounts`

**Fields**:
- `name`: CharField
- `slug`: SlugField (unique)
- `owner`: ForeignKey to User
- `stripe_customer_id`: CharField (optional)
- `plan`: ForeignKey to Plan
- `credits`: IntegerField (default: 0)
- `status`: CharField (choices: active, suspended, trial, cancelled)

**Methods**:
- `is_system_account()`: Returns True if account is system account (aws-admin, default-account, default)

#### User
**File**: `auth/models.py`

**Table**: `igny8_users`

**Fields**:
- Inherits from `AbstractUser`
- `email`: EmailField (unique, USERNAME_FIELD)
- `account`: ForeignKey to Account
- `role`: CharField (choices: developer, owner, admin, editor, viewer, system_bot)

**Methods**:
- `has_role(*roles)`: Check if user has any of the specified roles
- `is_owner_or_admin()`: Returns True if role is owner or admin
- `is_developer()`: Returns True if role is developer or is_superuser
- `is_admin_or_developer()`: Returns True if role is admin or developer
- `is_system_account_user()`: Returns True if user belongs to system account
- `get_accessible_sites()`: Returns QuerySet of sites user can access

#### Plan
**File**: `auth/models.py`

**Table**: `igny8_plans`

**Fields**:
- `name`: CharField
- `slug`: SlugField (unique)
- `price`: DecimalField
- `billing_cycle`: CharField (choices: monthly, annual)
- `features`: JSONField (array of feature strings)
- `max_users`: IntegerField
- `max_sites`: IntegerField
- `max_keywords`: IntegerField
- `max_clusters`: IntegerField
- `max_content_ideas`: IntegerField
- `daily_cluster_limit`: IntegerField
- `monthly_cluster_ai_credits`: IntegerField
- `daily_content_tasks`: IntegerField
- `daily_ai_requests`: IntegerField
- `monthly_word_count_limit`: IntegerField
- `monthly_content_ai_credits`: IntegerField
- `monthly_image_count`: IntegerField
- `daily_image_generation_limit`: IntegerField
- `monthly_image_ai_credits`: IntegerField
- `max_images_per_task`: IntegerField
- `image_model_choices`: JSONField
- `included_credits`: IntegerField
- `extra_credit_price`: DecimalField
- And more...

**Methods**:
- `clean()`: Validates plan limits
- `get_effective_credits_per_month()`: Returns included_credits or credits_per_month

#### Site
**File**: `auth/models.py`

**Table**: `igny8_sites`

**Fields**:
- Inherits from `AccountBaseModel`
- `name`: CharField
- `slug`: SlugField (unique per account)
- `domain`: URLField (optional)
- `industry`: ForeignKey to Industry (optional)
- `is_active`: BooleanField
- `status`: CharField (choices: active, inactive, suspended)
- `wp_url`: URLField (optional, WordPress integration)
- `wp_username`: CharField (optional)
- `wp_app_password`: CharField (optional)

**Methods**:
- `get_active_sectors_count()`: Get count of active sectors
- `can_add_sector()`: Check if site can add another sector (max 5)

#### Sector
**File**: `auth/models.py`

**Table**: `igny8_sectors`

**Fields**:
- Inherits from `AccountBaseModel`
- `site`: ForeignKey to Site
- `industry_sector`: ForeignKey to IndustrySector (optional, template reference)
- `name`: CharField
- `slug`: SlugField (unique per site)
- `is_active`: BooleanField
- `status`: CharField (choices: active, inactive)

### Planner Models

#### Keywords
**File**: `modules/planner/models.py`

**Table**: `igny8_keywords`

**Fields**:
- Inherits from `SiteSectorBaseModel`
- `keyword`: CharField
- `volume`: IntegerField
- `difficulty`: IntegerField
- `intent`: CharField (choices: informational, navigational, commercial, transactional)
- `cluster`: ForeignKey to Clusters (optional)
- `status`: CharField (choices: active, pending, archived)

#### Clusters
**File**: `modules/planner/models.py`

**Table**: `igny8_clusters`

**Fields**:
- Inherits from `SiteSectorBaseModel`
- `name`: CharField (unique)
- `description`: TextField
- `keywords_count`: IntegerField
- `volume`: IntegerField
- `mapped_pages`: IntegerField
- `status`: CharField

#### ContentIdeas
**File**: `modules/planner/models.py`

**Table**: `igny8_content_ideas`

**Fields**:
- Inherits from `SiteSectorBaseModel`
- `idea_title`: CharField
- `description`: TextField
- `content_structure`: CharField (choices: cluster_hub, landing_page, pillar_page, supporting_page)
- `content_type`: CharField (choices: blog_post, article, guide, tutorial)
- `target_keywords`: CharField (comma-separated, legacy)
- `keyword_objects`: ManyToManyField to Keywords
- `keyword_cluster`: ForeignKey to Clusters
- `status`: CharField (choices: new, scheduled, published)
- `estimated_word_count`: IntegerField

### Writer Models

#### Tasks
**File**: `modules/writer/models.py`

**Table**: `igny8_tasks`

**Fields**:
- Inherits from `SiteSectorBaseModel`
- `title`: CharField
- `description`: TextField
- `keywords`: CharField (comma-separated, legacy)
- `keyword_objects`: ManyToManyField to Keywords
- `cluster`: ForeignKey to Clusters
- `idea`: ForeignKey to ContentIdeas
- `content_structure`: CharField
- `content_type`: CharField
- `status`: CharField (choices: queued, in_progress, draft, review, published, completed)
- `content`: TextField (generated content)
- `word_count`: IntegerField
- `meta_title`: CharField
- `meta_description`: TextField
- `assigned_post_id`: IntegerField (WordPress post ID)
- `post_url`: URLField

#### Content
**File**: `modules/writer/models.py`

**Table**: `igny8_content`

**Fields**:
- Inherits from `SiteSectorBaseModel`
- `task`: OneToOneField to Tasks
- `html_content`: TextField
- `word_count`: IntegerField
- `metadata`: JSONField

**Methods**:
- `save()`: Automatically sets account, site, sector from task

#### Images
**File**: `modules/writer/models.py`

**Table**: `igny8_images`

**Fields**:
- Inherits from `SiteSectorBaseModel`
- `task`: ForeignKey to Tasks
- `image_type`: CharField (choices: featured, desktop, mobile, in_article)
- `image_url`: URLField
- `image_path`: CharField (local path)
- `prompt`: TextField
- `status`: CharField
- `position`: IntegerField

**Methods**:
- `save()`: Automatically sets account, site, sector from task

### System Models

#### AIPrompt
**File**: `modules/system/models.py`

**Table**: `igny8_ai_prompts`

**Fields**:
- Inherits from `AccountBaseModel`
- `prompt_type`: CharField (choices: clustering, ideas, content_generation, image_prompt_extraction, image_prompt_template, negative_prompt)
- `prompt_value`: TextField
- `default_prompt`: TextField
- `is_active`: BooleanField

**Unique Constraint**: `(account, prompt_type)`

#### IntegrationSettings
**File**: `modules/system/models.py`

**Table**: `igny8_integration_settings`

**Fields**:
- Inherits from `AccountBaseModel`
- `integration_type`: CharField (choices: openai, runware, gsc, image_generation)
- `config`: JSONField (API keys, settings, etc.)
- `is_active`: BooleanField

**Unique Constraint**: `(account, integration_type)`

#### AuthorProfile
**File**: `modules/system/models.py`

**Table**: `igny8_author_profiles`

**Fields**:
- Inherits from `AccountBaseModel`
- `name`: CharField
- `description`: TextField
- `tone`: CharField
- `language`: CharField
- `structure_template`: JSONField
- `is_active`: BooleanField

#### Strategy
**File**: `modules/system/models.py`

**Table**: `igny8_strategies`

**Fields**:
- Inherits from `AccountBaseModel`
- `name`: CharField
- `description`: TextField
- `sector`: ForeignKey to Sector (optional)
- `prompt_types`: JSONField
- `section_logic`: JSONField
- `is_active`: BooleanField

---

## ViewSets

### Base ViewSets

#### AccountModelViewSet
**File**: `api/base.py`

**Purpose**: Base ViewSet with automatic account filtering.

**Methods**:
- `get_queryset()`: Filters queryset by `request.account` (with admin/developer override)
- `perform_create()`: Sets account on created objects
- `get_serializer_context()`: Adds account to serializer context

**Access Control**:
- Admin/Developer users: Bypass account filtering
- System account users: Bypass account filtering
- Regular users: Only see data from their account

#### SiteSectorModelViewSet
**File**: `api/base.py`

**Purpose**: Base ViewSet with site/sector filtering and access control.

**Inherits**: `AccountModelViewSet`

**Methods**:
- `get_queryset()`: Filters by account, accessible sites, and optional site_id/sector_id
- `perform_create()`: Validates site access and sector-site relationship
- `get_serializer_context()`: Adds accessible sites and sectors to context

**Access Control**:
- Developers: All active sites
- System account users: All active sites
- Owners/Admins: All sites in their account
- Editors/Viewers: Only sites granted via `SiteUserAccess`

### Planner ViewSets

#### KeywordViewSet
**File**: `modules/planner/views.py`

**Inherits**: `SiteSectorModelViewSet`

**Actions**:
- `list()`: List keywords with filtering
- `create()`: Create keyword
- `retrieve()`: Get keyword details
- `update()`: Update keyword
- `destroy()`: Delete keyword
- `auto_cluster()`: Auto-cluster keywords using AI
- `bulk_delete()`: Bulk delete keywords
- `bulk_update_status()`: Bulk update keyword status
- `export_csv()`: Export keywords to CSV
- `import_csv()`: Import keywords from CSV

**Filtering**:
- Search: `keyword` field
- Filters: `status`, `cluster_id`, `intent`
- Custom: `difficulty_min`, `difficulty_max`, `volume_min`, `volume_max`
- Ordering: `created_at`, `volume`, `difficulty`

#### ClusterViewSet
**File**: `modules/planner/views.py`

**Inherits**: `SiteSectorModelViewSet`

**Actions**:
- `list()`: List clusters
- `create()`: Create cluster
- `retrieve()`: Get cluster details
- `update()`: Update cluster
- `destroy()`: Delete cluster
- `auto_generate_ideas()`: Auto-generate content ideas for clusters

#### ContentIdeasViewSet
**File**: `modules/planner/views.py`

**Inherits**: `SiteSectorModelViewSet`

**Actions**:
- `list()`: List content ideas
- `create()`: Create content idea
- `retrieve()`: Get content idea details
- `update()`: Update content idea
- `destroy()`: Delete content idea

### Writer ViewSets

#### TasksViewSet
**File**: `modules/writer/views.py`

**Inherits**: `SiteSectorModelViewSet`

**Actions**:
- `list()`: List tasks
- `create()`: Create task
- `retrieve()`: Get task details
- `update()`: Update task
- `destroy()`: Delete task
- `auto_generate_content()`: Auto-generate content for tasks
- `auto_generate_images()`: Auto-generate images for tasks
- `bulk_delete()`: Bulk delete tasks
- `bulk_update()`: Bulk update task status

**Filtering**:
- Search: `title`, `keywords`
- Filters: `status`, `cluster_id`, `content_type`, `content_structure`
- Ordering: `title`, `created_at`, `word_count`, `status`

### System ViewSets

#### IntegrationSettingsViewSet
**File**: `modules/system/integration_views.py`

**Inherits**: `viewsets.ViewSet`

**Actions**:
- `list()`: List integrations
- `retrieve()`: Get integration settings
- `update()`: Save integration settings
- `save_post()`: Save integration settings (POST)
- `test_connection()`: Test API connection
- `test_openai()`: Test OpenAI connection
- `test_runware()`: Test Runware connection
- `generate_image()`: Test image generation
- `task_progress()`: Get Celery task progress

#### AIPromptViewSet
**File**: `modules/system/views.py`

**Inherits**: `AccountModelViewSet`

**Actions**:
- `list()`: List prompts
- `create()`: Create prompt
- `retrieve()`: Get prompt details
- `update()`: Update prompt
- `destroy()`: Delete prompt
- `reset_to_default()`: Reset prompt to default value

#### AuthorProfileViewSet
**File**: `modules/system/views.py`

**Inherits**: `AccountModelViewSet`

**Actions**:
- `list()`: List author profiles
- `create()`: Create author profile
- `retrieve()`: Get author profile details
- `update()`: Update author profile
- `destroy()`: Delete author profile

---

## Serializers

### Planner Serializers

#### KeywordSerializer
**File**: `modules/planner/serializers.py`

**Fields**: All Keyword model fields

**Validation**: Validates keyword uniqueness, cluster belongs to same sector

#### ClusterSerializer
**File**: `modules/planner/cluster_serializers.py`

**Fields**: All Cluster model fields

**Read-Only Fields**: `keywords_count`, `volume` (calculated)

#### ContentIdeasSerializer
**File**: `modules/planner/serializers.py`

**Fields**: All ContentIdeas model fields

### Writer Serializers

#### TasksSerializer
**File**: `modules/writer/serializers.py`

**Fields**: All Tasks model fields

#### ContentSerializer
**File**: `modules/writer/serializers.py`

**Fields**: All Content model fields

#### ImagesSerializer
**File**: `modules/writer/serializers.py`

**Fields**: All Images model fields

### System Serializers

#### AIPromptSerializer
**File**: `modules/system/serializers.py`

**Fields**: All AIPrompt model fields

#### IntegrationSettingsSerializer
**File**: `modules/system/serializers.py`

**Fields**: All IntegrationSettings model fields

---

## Celery Tasks

### Planner Tasks

#### auto_cluster_keywords_task
**File**: `modules/planner/tasks.py`

**Purpose**: Auto-cluster keywords using AI.

**Parameters**:
- `keyword_ids`: List of keyword IDs
- `account_id`: Account ID
- `site_id`: Site ID
- `sector_id`: Sector ID

**Progress Tracking**: Updates progress with request_steps and response_steps

**Calls**: `_auto_cluster_keywords_core()`

#### auto_generate_ideas_task
**File**: `modules/planner/tasks.py`

**Purpose**: Auto-generate content ideas for clusters.

**Parameters**:
- `cluster_ids`: List of cluster IDs
- `account_id`: Account ID

**Progress Tracking**: Updates progress for each cluster

**Calls**: `_generate_single_idea_core()` for each cluster

### Writer Tasks

#### auto_generate_content_task
**File**: `modules/writer/tasks.py`

**Purpose**: Auto-generate content for tasks.

**Parameters**:
- `task_ids`: List of task IDs
- `account_id`: Account ID

**Progress Tracking**: Updates progress for each task

**Calls**: `AIProcessor.generate_content()`

#### auto_generate_images_task
**File**: `modules/writer/tasks.py`

**Purpose**: Auto-generate images for tasks.

**Parameters**:
- `task_ids`: List of task IDs
- `account_id`: Account ID

**Progress Tracking**: Updates progress for each task

**Calls**: `AIProcessor.extract_image_prompts()` and `AIProcessor.generate_image()`

---

## API Endpoints

### Base URL

`/api/v1/`

### Planner Endpoints

- `GET /api/v1/planner/keywords/` - List keywords
- `POST /api/v1/planner/keywords/` - Create keyword
- `GET /api/v1/planner/keywords/{id}/` - Get keyword
- `PUT /api/v1/planner/keywords/{id}/` - Update keyword
- `DELETE /api/v1/planner/keywords/{id}/` - Delete keyword
- `POST /api/v1/planner/keywords/auto_cluster/` - Auto-cluster keywords
- `POST /api/v1/planner/keywords/bulk_delete/` - Bulk delete keywords
- `POST /api/v1/planner/keywords/bulk_update_status/` - Bulk update status
- `GET /api/v1/planner/keywords/export_csv/` - Export keywords
- `POST /api/v1/planner/keywords/import_csv/` - Import keywords
- `GET /api/v1/planner/clusters/` - List clusters
- `POST /api/v1/planner/clusters/auto_generate_ideas/` - Auto-generate ideas
- `GET /api/v1/planner/ideas/` - List content ideas

### Writer Endpoints

- `GET /api/v1/writer/tasks/` - List tasks
- `POST /api/v1/writer/tasks/auto_generate_content/` - Auto-generate content
- `POST /api/v1/writer/tasks/auto_generate_images/` - Auto-generate images

### System Endpoints

- `GET /api/v1/system/settings/integrations/{pk}/` - Get integration settings
- `PUT /api/v1/system/settings/integrations/{pk}/` - Save integration settings
- `POST /api/v1/system/settings/integrations/{pk}/test_openai/` - Test OpenAI
- `POST /api/v1/system/settings/integrations/{pk}/test_runware/` - Test Runware
- `POST /api/v1/system/settings/integrations/{pk}/generate_image/` - Test image generation
- `GET /api/v1/system/settings/task_progress/{task_id}/` - Get task progress

---

## Base Classes

### AccountModelViewSet

**File**: `api/base.py`

**Purpose**: Base ViewSet with automatic account filtering.

**Features**:
- Automatic account filtering
- Admin/Developer override
- Account context in serializers

### SiteSectorModelViewSet

**File**: `api/base.py`

**Purpose**: Base ViewSet with site/sector filtering.

**Features**:
- Account filtering (inherited)
- Site access control
- Sector validation
- Accessible sites/sectors in serializer context

---

## Middleware

### AccountContextMiddleware

**File**: `middleware/account.py`

**Purpose**: Sets `request.account` from JWT token.

**Functionality**:
- Extracts account ID from JWT token
- Loads Account object
- Sets `request.account`

### ResourceTrackerMiddleware

**File**: `middleware/resource_tracker.py`

**Purpose**: Tracks API request metrics.

**Functionality**:
- Tracks CPU, memory, I/O usage
- Stores metrics in cache
- Provides metrics endpoint

---

## Utilities

### AIProcessor

**File**: `utils/ai_processor.py`

**Purpose**: Unified AI interface for all AI operations.

**Methods**:
- `cluster_keywords()`: Cluster keywords using AI
- `generate_ideas()`: Generate content ideas
- `generate_content()`: Generate text content
- `extract_image_prompts()`: Extract image prompts from content
- `generate_image()`: Generate images using OpenAI DALL-E or Runware

**See**: AI Functions documentation for complete details

### Content Normalizer

**File**: `utils/content_normalizer.py`

**Purpose**: Content processing utilities.

**Functions**:
- `_extract_body_content()`: Extract body content from HTML

---

## Modules

### Planner Module

**Purpose**: Keyword management and content planning.

**Models**: Keywords, Clusters, ContentIdeas

**ViewSets**: KeywordViewSet, ClusterViewSet, ContentIdeasViewSet

**Tasks**: auto_cluster_keywords_task, auto_generate_ideas_task

### Writer Module

**Purpose**: Content generation and management.

**Models**: Tasks, Content, Images

**ViewSets**: TasksViewSet

**Tasks**: auto_generate_content_task, auto_generate_images_task

### System Module

**Purpose**: System settings, prompts, and integrations.

**Models**: AIPrompt, IntegrationSettings, AuthorProfile, Strategy

**ViewSets**: AIPromptViewSet, AuthorProfileViewSet, IntegrationSettingsViewSet

**Utilities**: Default prompts, prompt loading

### Billing Module

**Purpose**: Credits, transactions, and usage tracking.

**Models**: CreditTransaction, UsageLog

**ViewSets**: CreditTransactionViewSet, UsageLogViewSet

**Services**: CreditService

---

## Summary

The IGNY8 backend is built on:

1. **Django + DRF**: Robust web framework with RESTful API
2. **Multi-Tenancy**: Complete account isolation with automatic filtering
3. **Modular Architecture**: Clear module boundaries with shared utilities
4. **Celery Integration**: Asynchronous task processing for long-running operations
5. **Base ViewSets**: Consistent access control and filtering
6. **AI Integration**: Unified AIProcessor for all AI operations
7. **Progress Tracking**: Real-time progress updates for Celery tasks
8. **Account/Site/Sector Hierarchy**: Hierarchical data organization

This architecture ensures scalability, maintainability, and extensibility while providing a robust API for the frontend.