igny8/ActiveDocs/03-BACKEND.md

IGNY8 Backend Documentation

Version: 1.0
Last Updated: 2025-01-XX
Purpose: Complete backend documentation covering models, views, APIs, modules, serializers, tasks, and structure.

---

Table of Contents

1. Backend Overview
2. Tech Stack
3. Project Structure
4. Models
5. ViewSets
6. Serializers
7. Celery Tasks
8. API Endpoints
9. Base Classes
10. Middleware
11. Utilities
12. Modules

---

Backend Overview

The IGNY8 backend is a Django 5.2+ application using Django REST Framework (DRF) for API endpoints. The backend follows a modular architecture with clear separation of concerns, automatic account isolation, and support for asynchronous task processing via Celery.

Key Features

• Multi-Tenancy: Complete account isolation with automatic filtering
• RESTful API: DRF ViewSets with consistent response format
• Celery Integration: Asynchronous task processing for long-running operations
• Account/Site/Sector Hierarchy: Hierarchical data organization
• AI Integration: Unified AIProcessor for all AI operations
• Progress Tracking: Real-time progress updates for Celery tasks

---

Tech Stack

Core Technologies

• Django 5.2+: Web framework
• Django REST Framework: API framework
• PostgreSQL: Database
• Celery: Asynchronous task queue
• Redis: Celery broker and caching

Key Libraries

• django-filter: Advanced filtering
• djangorestframework-simplejwt: JWT authentication
• requests: HTTP client for external APIs
• python-dotenv: Environment variable management

---

Project Structure

backend/igny8_core/
├── auth/                     # Multi-tenancy and authentication
│   ├── models.py             # Account, User, Plan, Site, Sector, Industry models
│   ├── views.py              # Account, User, Site, Sector ViewSets
│   ├── serializers.py        # Account, User, Plan serializers
│   └── urls.py               # Auth module URLs
├── modules/                  # Feature modules
│   ├── planner/              # Keywords, Clusters, Ideas
│   │   ├── models.py         # Keywords, Clusters, ContentIdeas models
│   │   ├── views.py          # KeywordViewSet, ClusterViewSet, ContentIdeasViewSet
│   │   ├── tasks.py          # Celery tasks for AI operations
│   │   ├── serializers.py    # Model serializers
│   │   └── urls.py           # Planner module URLs
│   ├── writer/               # Tasks, Content, Images
│   │   ├── models.py         # Tasks, Content, Images models
│   │   ├── views.py          # TasksViewSet
│   │   ├── tasks.py          # Celery tasks for content/image generation
│   │   └── urls.py           # Writer module URLs
│   ├── system/               # Settings, Prompts, Integration
│   │   ├── models.py         # AIPrompt, IntegrationSettings, AuthorProfile, Strategy
│   │   ├── views.py          # AIPromptViewSet, AuthorProfileViewSet
│   │   ├── integration_views.py  # IntegrationSettingsViewSet, task_progress
│   │   ├── utils.py          # Default prompts, prompt loading
│   │   └── urls.py           # System module URLs
│   └── billing/              # Credits, Transactions, Usage
│       ├── models.py         # CreditTransaction, UsageLog models
│       ├── views.py          # Billing ViewSets
│       └── services.py       # CreditService
├── api/                      # API base classes
│   ├── base.py               # AccountModelViewSet, SiteSectorModelViewSet
│   └── pagination.py         # CustomPageNumberPagination
├── utils/                    # Shared utilities
│   ├── ai_processor.py       # Unified AI interface
│   └── content_normalizer.py # Content processing utilities
├── middleware/               # Custom middleware
│   ├── account.py            # AccountContextMiddleware (sets request.account)
│   └── resource_tracker.py   # ResourceTrackerMiddleware (API metrics)
├── settings.py               # Django settings
├── urls.py                   # Root URL configuration
└── celery.py                 # Celery configuration

---

Models

Base Models

AccountBaseModel

File: auth/models.py

Purpose: Base model for all account-isolated models.

Fields:

• account: ForeignKey to Account
• created_at: DateTimeField (auto_now_add)
• updated_at: DateTimeField (auto_now)

Usage: All models that need account isolation inherit from this.

SiteSectorBaseModel

File: auth/models.py

Purpose: Base model for models that belong to Site and Sector.

Fields:

• Inherits from AccountBaseModel
• site: ForeignKey to Site
• sector: ForeignKey to Sector

Methods:

• save(): Automatically sets account from site.account and validates sector belongs to site

Usage: Models like Keywords, Clusters, ContentIdeas, Tasks inherit from this.

Auth Models

Account

File: auth/models.py

Table: igny8_accounts

Fields:

• name: CharField
• slug: SlugField (unique)
• owner: ForeignKey to User
• stripe_customer_id: CharField (optional)
• plan: ForeignKey to Plan
• credits: IntegerField (default: 0)
• status: CharField (choices: active, suspended, trial, cancelled)

Methods:

• is_system_account(): Returns True if account is system account (aws-admin, default-account, default)

User

File: auth/models.py

Table: igny8_users

Fields:

• Inherits from AbstractUser
• email: EmailField (unique, USERNAME_FIELD)
• account: ForeignKey to Account
• role: CharField (choices: developer, owner, admin, editor, viewer, system_bot)

Methods:

• has_role(*roles): Check if user has any of the specified roles
• is_owner_or_admin(): Returns True if role is owner or admin
• is_developer(): Returns True if role is developer or is_superuser
• is_admin_or_developer(): Returns True if role is admin or developer
• is_system_account_user(): Returns True if user belongs to system account
• get_accessible_sites(): Returns QuerySet of sites user can access

Plan

File: auth/models.py

Table: igny8_plans

Fields:

• name: CharField
• slug: SlugField (unique)
• price: DecimalField
• billing_cycle: CharField (choices: monthly, annual)
• features: JSONField (array of feature strings)
• max_users: IntegerField
• max_sites: IntegerField
• max_keywords: IntegerField
• max_clusters: IntegerField
• max_content_ideas: IntegerField
• daily_cluster_limit: IntegerField
• monthly_cluster_ai_credits: IntegerField
• daily_content_tasks: IntegerField
• daily_ai_requests: IntegerField
• monthly_word_count_limit: IntegerField
• monthly_content_ai_credits: IntegerField
• monthly_image_count: IntegerField
• daily_image_generation_limit: IntegerField
• monthly_image_ai_credits: IntegerField
• max_images_per_task: IntegerField
• image_model_choices: JSONField
• included_credits: IntegerField
• extra_credit_price: DecimalField
• And more...

Methods:

• clean(): Validates plan limits
• get_effective_credits_per_month(): Returns included_credits or credits_per_month

Site

File: auth/models.py

Table: igny8_sites

Fields:

• Inherits from AccountBaseModel
• name: CharField
• slug: SlugField (unique per account)
• domain: URLField (optional)
• industry: ForeignKey to Industry (optional)
• is_active: BooleanField
• status: CharField (choices: active, inactive, suspended)
• wp_url: URLField (optional, WordPress integration)
• wp_username: CharField (optional)
• wp_app_password: CharField (optional)

Methods:

• get_active_sectors_count(): Get count of active sectors
• can_add_sector(): Check if site can add another sector (max 5)

Sector

File: auth/models.py

Table: igny8_sectors

Fields:

• Inherits from AccountBaseModel
• site: ForeignKey to Site
• industry_sector: ForeignKey to IndustrySector (optional, template reference)
• name: CharField
• slug: SlugField (unique per site)
• is_active: BooleanField
• status: CharField (choices: active, inactive)

Planner Models

Keywords

File: modules/planner/models.py

Table: igny8_keywords

Fields:

• Inherits from SiteSectorBaseModel
• keyword: CharField
• volume: IntegerField
• difficulty: IntegerField
• intent: CharField (choices: informational, navigational, commercial, transactional)
• cluster: ForeignKey to Clusters (optional)
• status: CharField (choices: active, pending, archived)

Clusters

File: modules/planner/models.py

Table: igny8_clusters

Fields:

• Inherits from SiteSectorBaseModel
• name: CharField (unique)
• description: TextField
• keywords_count: IntegerField
• volume: IntegerField
• mapped_pages: IntegerField
• status: CharField

ContentIdeas

File: modules/planner/models.py

Table: igny8_content_ideas

Fields:

• Inherits from SiteSectorBaseModel
• idea_title: CharField
• description: TextField
• content_structure: CharField (choices: cluster_hub, landing_page, pillar_page, supporting_page)
• content_type: CharField (choices: blog_post, article, guide, tutorial)
• target_keywords: CharField (comma-separated, legacy)
• keyword_objects: ManyToManyField to Keywords
• keyword_cluster: ForeignKey to Clusters
• status: CharField (choices: new, scheduled, published)
• estimated_word_count: IntegerField

Writer Models

Tasks

File: modules/writer/models.py

Table: igny8_tasks

Fields:

• Inherits from SiteSectorBaseModel
• title: CharField
• description: TextField
• keywords: CharField (comma-separated, legacy)
• keyword_objects: ManyToManyField to Keywords
• cluster: ForeignKey to Clusters
• idea: ForeignKey to ContentIdeas
• content_structure: CharField
• content_type: CharField
• status: CharField (choices: queued, in_progress, draft, review, published, completed)
• content: TextField (generated content)
• word_count: IntegerField
• meta_title: CharField
• meta_description: TextField
• assigned_post_id: IntegerField (WordPress post ID)
• post_url: URLField

Content

File: modules/writer/models.py

Table: igny8_content

Fields:

• Inherits from SiteSectorBaseModel
• task: OneToOneField to Tasks
• html_content: TextField
• word_count: IntegerField
• metadata: JSONField

Methods:

• save(): Automatically sets account, site, sector from task

Images

File: modules/writer/models.py

Table: igny8_images

Fields:

• Inherits from SiteSectorBaseModel
• task: ForeignKey to Tasks
• image_type: CharField (choices: featured, desktop, mobile, in_article)
• image_url: URLField
• image_path: CharField (local path)
• prompt: TextField
• status: CharField
• position: IntegerField

Methods:

• save(): Automatically sets account, site, sector from task

System Models

AIPrompt

File: modules/system/models.py

Table: igny8_ai_prompts

Fields:

• Inherits from AccountBaseModel
• prompt_type: CharField (choices: clustering, ideas, content_generation, image_prompt_extraction, image_prompt_template, negative_prompt)
• prompt_value: TextField
• default_prompt: TextField
• is_active: BooleanField

Unique Constraint: (account, prompt_type)

IntegrationSettings

File: modules/system/models.py

Table: igny8_integration_settings

Fields:

• Inherits from AccountBaseModel
• integration_type: CharField (choices: openai, runware, gsc, image_generation)
• config: JSONField (API keys, settings, etc.)
• is_active: BooleanField

Unique Constraint: (account, integration_type)

AuthorProfile

File: modules/system/models.py

Table: igny8_author_profiles

Fields:

• Inherits from AccountBaseModel
• name: CharField
• description: TextField
• tone: CharField
• language: CharField
• structure_template: JSONField
• is_active: BooleanField

Strategy

File: modules/system/models.py

Table: igny8_strategies

Fields:

• Inherits from AccountBaseModel
• name: CharField
• description: TextField
• sector: ForeignKey to Sector (optional)
• prompt_types: JSONField
• section_logic: JSONField
• is_active: BooleanField

---

ViewSets

Base ViewSets

AccountModelViewSet

File: api/base.py

Purpose: Base ViewSet with automatic account filtering.

Methods:

• get_queryset(): Filters queryset by request.account (with admin/developer override)
• perform_create(): Sets account on created objects
• get_serializer_context(): Adds account to serializer context

Access Control:

• Admin/Developer users: Bypass account filtering
• System account users: Bypass account filtering
• Regular users: Only see data from their account

SiteSectorModelViewSet

File: api/base.py

Purpose: Base ViewSet with site/sector filtering and access control.

Inherits: AccountModelViewSet

Methods:

• get_queryset(): Filters by account, accessible sites, and optional site_id/sector_id
• perform_create(): Validates site access and sector-site relationship
• get_serializer_context(): Adds accessible sites and sectors to context

Access Control:

• Developers: All active sites
• System account users: All active sites
• Owners/Admins: All sites in their account
• Editors/Viewers: Only sites granted via SiteUserAccess

Planner ViewSets

KeywordViewSet

File: modules/planner/views.py

Inherits: SiteSectorModelViewSet

Actions:

• list(): List keywords with filtering
• create(): Create keyword
• retrieve(): Get keyword details
• update(): Update keyword
• destroy(): Delete keyword
• auto_cluster(): Auto-cluster keywords using AI
• bulk_delete(): Bulk delete keywords
• bulk_update_status(): Bulk update keyword status
• export_csv(): Export keywords to CSV
• import_csv(): Import keywords from CSV

Filtering:

• Search: keyword field
• Filters: status, cluster_id, intent
• Custom: difficulty_min, difficulty_max, volume_min, volume_max
• Ordering: created_at, volume, difficulty

ClusterViewSet

File: modules/planner/views.py

Inherits: SiteSectorModelViewSet

Actions:

• list(): List clusters
• create(): Create cluster
• retrieve(): Get cluster details
• update(): Update cluster
• destroy(): Delete cluster
• auto_generate_ideas(): Auto-generate content ideas for clusters

ContentIdeasViewSet

File: modules/planner/views.py

Inherits: SiteSectorModelViewSet

Actions:

• list(): List content ideas
• create(): Create content idea
• retrieve(): Get content idea details
• update(): Update content idea
• destroy(): Delete content idea

Writer ViewSets

TasksViewSet

File: modules/writer/views.py

Inherits: SiteSectorModelViewSet

Actions:

• list(): List tasks
• create(): Create task
• retrieve(): Get task details
• update(): Update task
• destroy(): Delete task
• auto_generate_content(): Auto-generate content for tasks
• auto_generate_images(): Auto-generate images for tasks
• bulk_delete(): Bulk delete tasks
• bulk_update(): Bulk update task status

Filtering:

• Search: title, keywords
• Filters: status, cluster_id, content_type, content_structure
• Ordering: title, created_at, word_count, status

System ViewSets

IntegrationSettingsViewSet

File: modules/system/integration_views.py

Inherits: viewsets.ViewSet

Actions:

• list(): List integrations
• retrieve(): Get integration settings
• update(): Save integration settings
• save_post(): Save integration settings (POST)
• test_connection(): Test API connection
• test_openai(): Test OpenAI connection
• test_runware(): Test Runware connection
• generate_image(): Test image generation
• task_progress(): Get Celery task progress

AIPromptViewSet

File: modules/system/views.py

Inherits: AccountModelViewSet

Actions:

• list(): List prompts
• create(): Create prompt
• retrieve(): Get prompt details
• update(): Update prompt
• destroy(): Delete prompt
• reset_to_default(): Reset prompt to default value

AuthorProfileViewSet

File: modules/system/views.py

Inherits: AccountModelViewSet

Actions:

• list(): List author profiles
• create(): Create author profile
• retrieve(): Get author profile details
• update(): Update author profile
• destroy(): Delete author profile

---

Serializers

Planner Serializers

KeywordSerializer

File: modules/planner/serializers.py

Fields: All Keyword model fields

Validation: Validates keyword uniqueness, cluster belongs to same sector

ClusterSerializer

File: modules/planner/cluster_serializers.py

Fields: All Cluster model fields

Read-Only Fields: keywords_count, volume (calculated)

ContentIdeasSerializer

File: modules/planner/serializers.py

Fields: All ContentIdeas model fields

Writer Serializers

TasksSerializer

File: modules/writer/serializers.py

Fields: All Tasks model fields

ContentSerializer

File: modules/writer/serializers.py

Fields: All Content model fields

ImagesSerializer

File: modules/writer/serializers.py

Fields: All Images model fields

System Serializers

AIPromptSerializer

File: modules/system/serializers.py

Fields: All AIPrompt model fields

IntegrationSettingsSerializer

File: modules/system/serializers.py

Fields: All IntegrationSettings model fields

---

Celery Tasks

Planner Tasks

auto_cluster_keywords_task

File: modules/planner/tasks.py

Purpose: Auto-cluster keywords using AI.

Parameters:

• keyword_ids: List of keyword IDs
• account_id: Account ID
• site_id: Site ID
• sector_id: Sector ID

Progress Tracking: Updates progress with request_steps and response_steps

Calls: _auto_cluster_keywords_core()

auto_generate_ideas_task

File: modules/planner/tasks.py

Purpose: Auto-generate content ideas for clusters.

Parameters:

• cluster_ids: List of cluster IDs
• account_id: Account ID

Progress Tracking: Updates progress for each cluster

Calls: _generate_single_idea_core() for each cluster

Writer Tasks

auto_generate_content_task

File: modules/writer/tasks.py

Purpose: Auto-generate content for tasks.

Parameters:

• task_ids: List of task IDs
• account_id: Account ID

Progress Tracking: Updates progress for each task

Calls: AIProcessor.generate_content()

auto_generate_images_task

File: modules/writer/tasks.py

Purpose: Auto-generate images for tasks.

Parameters:

• task_ids: List of task IDs
• account_id: Account ID

Progress Tracking: Updates progress for each task

Calls: AIProcessor.extract_image_prompts() and AIProcessor.generate_image()

---

API Endpoints

Base URL

/api/v1/

Planner Endpoints

• GET /api/v1/planner/keywords/ - List keywords
• POST /api/v1/planner/keywords/ - Create keyword
• GET /api/v1/planner/keywords/{id}/ - Get keyword
• PUT /api/v1/planner/keywords/{id}/ - Update keyword
• DELETE /api/v1/planner/keywords/{id}/ - Delete keyword
• POST /api/v1/planner/keywords/auto_cluster/ - Auto-cluster keywords
• POST /api/v1/planner/keywords/bulk_delete/ - Bulk delete keywords
• POST /api/v1/planner/keywords/bulk_update_status/ - Bulk update status
• GET /api/v1/planner/keywords/export_csv/ - Export keywords
• POST /api/v1/planner/keywords/import_csv/ - Import keywords
• GET /api/v1/planner/clusters/ - List clusters
• POST /api/v1/planner/clusters/auto_generate_ideas/ - Auto-generate ideas
• GET /api/v1/planner/ideas/ - List content ideas

Writer Endpoints

• GET /api/v1/writer/tasks/ - List tasks
• POST /api/v1/writer/tasks/auto_generate_content/ - Auto-generate content
• POST /api/v1/writer/tasks/auto_generate_images/ - Auto-generate images

System Endpoints

• GET /api/v1/system/settings/integrations/{pk}/ - Get integration settings
• PUT /api/v1/system/settings/integrations/{pk}/ - Save integration settings
• POST /api/v1/system/settings/integrations/{pk}/test_openai/ - Test OpenAI
• POST /api/v1/system/settings/integrations/{pk}/test_runware/ - Test Runware
• POST /api/v1/system/settings/integrations/{pk}/generate_image/ - Test image generation
• GET /api/v1/system/settings/task_progress/{task_id}/ - Get task progress

---

Base Classes

AccountModelViewSet

File: api/base.py

Purpose: Base ViewSet with automatic account filtering.

Features:

• Automatic account filtering
• Admin/Developer override
• Account context in serializers

SiteSectorModelViewSet

File: api/base.py

Purpose: Base ViewSet with site/sector filtering.

Features:

• Account filtering (inherited)
• Site access control
• Sector validation
• Accessible sites/sectors in serializer context

---

Middleware

AccountContextMiddleware

File: middleware/account.py

Purpose: Sets request.account from JWT token.

Functionality:

• Extracts account ID from JWT token
• Loads Account object
• Sets request.account

ResourceTrackerMiddleware

File: middleware/resource_tracker.py

Purpose: Tracks API request metrics.

Functionality:

• Tracks CPU, memory, I/O usage
• Stores metrics in cache
• Provides metrics endpoint

---

Utilities

AIProcessor

File: utils/ai_processor.py

Purpose: Unified AI interface for all AI operations.

Methods:

• cluster_keywords(): Cluster keywords using AI
• generate_ideas(): Generate content ideas
• generate_content(): Generate text content
• extract_image_prompts(): Extract image prompts from content
• generate_image(): Generate images using OpenAI DALL-E or Runware

See: AI Functions documentation for complete details

Content Normalizer

File: utils/content_normalizer.py

Purpose: Content processing utilities.

Functions:

• _extract_body_content(): Extract body content from HTML

---

Modules

Planner Module

Purpose: Keyword management and content planning.

Models: Keywords, Clusters, ContentIdeas

ViewSets: KeywordViewSet, ClusterViewSet, ContentIdeasViewSet

Tasks: auto_cluster_keywords_task, auto_generate_ideas_task

Writer Module

Purpose: Content generation and management.

Models: Tasks, Content, Images

ViewSets: TasksViewSet

Tasks: auto_generate_content_task, auto_generate_images_task

System Module

Purpose: System settings, prompts, and integrations.

Models: AIPrompt, IntegrationSettings, AuthorProfile, Strategy

ViewSets: AIPromptViewSet, AuthorProfileViewSet, IntegrationSettingsViewSet

Utilities: Default prompts, prompt loading

Billing Module

Purpose: Credits, transactions, and usage tracking.

Models: CreditTransaction, UsageLog

ViewSets: CreditTransactionViewSet, UsageLogViewSet

Services: CreditService

---

Summary

The IGNY8 backend is built on:

1. Django + DRF: Robust web framework with RESTful API
2. Multi-Tenancy: Complete account isolation with automatic filtering
3. Modular Architecture: Clear module boundaries with shared utilities
4. Celery Integration: Asynchronous task processing for long-running operations
5. Base ViewSets: Consistent access control and filtering
6. AI Integration: Unified AIProcessor for all AI operations
7. Progress Tracking: Real-time progress updates for Celery tasks
8. Account/Site/Sector Hierarchy: Hierarchical data organization

This architecture ensures scalability, maintainability, and extensibility while providing a robust API for the frontend.